FACE AND FACIAL FEATURE TRACKING USING DEFORMABLE MODELS

Abstract

In this paper, we address the 3D tracking of pose and animation of the human face in monocular image sequences using deformable 3D models. The main contributions of this paper are as follows. First, we show how the robustness and stability of the Active Appearance Algorithm can be improved through the inclusion of a simple motion compensation based on feature correspondence. Second, we develop a new method able to adapt a deformable 3D model to a face in the input image. Central to this method is the decoupling of global head movements and local non-rigid deformations/animations. This decoupling is achieved by, first, estimating the global (rigid) motion using robust statistics and a statistical model for face texture, and then, adapting the 3D model to possible local animations using the concept of the Active Appearance Algorithm. This proposed method constitutes a significant step towards reliable model-based face trackers since the strengths of complementary tracking methodologies are combined. Experiments evaluating the effectiveness of the methods are reported. Adaptation and tracking examples demonstrate the feasibility and robustness of the developed methods.