INTELLIGENT UNDERSTANDING OF USER INTERACTION IN IMAGE SEGMENTATION

Abstract

We have developed interactive tools for graph-based segmentation of natural images, in which the user guides object delineation by drawing strokes (markers) inside and outside the object. A suitable arc-weight estimation is paramount to minimize user time and maximize segmentation accuracy in these tools. However, it depends on discriminative image properties for object and background. These properties can be obtained from some marker pixels, but their identification is a hard problem during delineation. Careless arc-weight re-estimation reduces user control and drops performance, while interactive arc-weight estimation in a step before interactive object extraction is the best option so far, albeit it is not intuitive for nonexpert users. We present an effective solution using the unified framework of the image foresting transform (IFT) with three operators: clustering for interpreting user interaction and determining when and where arc weights need to be re-estimated; fuzzy classification for arc-weight estimation; and marker competition based on optimum connectivity for object extraction. For validation, we compared the proposed approach with another interactive IFT-based method, which computes arc weights before extraction. Evaluation involved multiple users (experts and nonexperts), a dataset with several natural images, and measurements to quantify accuracy, precision, efficiency (user time and computation time), and user control, being some of them novel measurements, proposed in this work.