Multi-object tracking using deformable convolution networks with tracklets updating

Abstract

Many multi-object tracking methods have been proposed to solve the computer vision problem which has been attracting significant attentions because of the significant appearance changes caused by deformation, abrupt motion, background clutter and occlusion. In this paper, hybrid deformable convolution neural networks with frame-pair input and deformable layers for multi-object tracking are presented. The object tracking method trained using two successive frames can predict the centers of searching windows as the locations of tracked targets to improve the accuracy and robustness of object tracking. Histogram of Oriented Gradient and CNN features are extracted as appearance features to measure similarities between objects. Kalman filter and Hungarian algorithm are used to create tracklets association which indicates the location and the trajectories of tracked targets. To solve the problem of object transformation, we construct a novel sampling strategy for off-line training with the idea of augmenting the special sampling locations in the convolution layers and pooling layers with additional offsets. Experiments on the popular challenging datasets show that the proposed tracking system performs on par with recently developed generic multi-object tracking methods, but effective for dense geometric transformation objects and with much less memory. In addition, the proposed tracking system can run in a speed of over 75 (24) fps with a GPU (CPU), much faster than most deep networks-based trackers.