Joint Space-Time-Range Mean ShiftBased Image and Video Segmentation

Abstract

This chapter addresses image and video segmentation by using mean shift-based filtering and segmentation. Mean shift is an effective and elegant method to directly seek the local modes (or, local maxima) of the probability density function without the requirement of actually estimating it. Mean shift is proportional to the normalized density gradient estimate, and is pointing to the local stationary point (or, local mode) of the density estimate at which it converges. A mean shift filter can be related to a domain filter, a range filter or a bilateral filter depending on the variable setting in the kernel, and also has its own strength due to its flexibility and statistical basis. In this chapter a variety of mean shift filtering approaches are described for image/video segmentation and nonlinear edge-preserving image smoothing. A joint space-time-range domain mean shift-based video segmentation approach is presented. Segmentation of moving/static objects/background is obtained through inter-frame mode-matching in consecutive frames and motion vector mode estimation. Newly appearing objects/regions in the current frame due to new foreground objects or uncovered background regions are segmented by intra-frame mode estimation. Examples of image/video segmentation are included to demonstrate the effectiveness and robustness of these methods. Pseudo codes of the algorithms are also included.