Regularization for Unsupervised Learning of Optical Flow
Author:
Long Libo1, Lang Jochen1ORCID
Affiliation:
1. Faculty of Engineering, University of Ottawa, Ottawa, ON K1N 6N5, Canada
Abstract
Regularization is an important technique for training deep neural networks. In this paper, we propose a novel shared-weight teacher–student strategy and a content-aware regularization (CAR) module. Based on a tiny, learnable, content-aware mask, CAR is randomly applied to some channels in the convolutional layers during training to be able to guide predictions in a shared-weight teacher–student strategy. CAR prevents motion estimation methods in unsupervised learning from co-adaptation. Extensive experiments on optical flow and scene flow estimation show that our method significantly improves on the performance of the original networks and surpasses other popular regularization methods. The method also surpasses all variants with similar architectures and the supervised PWC-Net on MPI-Sintel and on KITTI. Our method shows strong cross-dataset generalization, i.e., our method solely trained on MPI-Sintel outperforms a similarly trained supervised PWC-Net by 27.9% and 32.9% on KITTI, respectively. Our method uses fewer parameters and less computation, and has faster inference times than the original PWC-Net.
Funder
Natural Sciences and Engineering Research Council of Canada
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference70 articles.
1. Jiang, H., Sun, D., Jampani, V., Yang, M.H., Learned-Miller, E., and Kautz, J. (2018, January 18–23). Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA. 2. Two-Stream Convolutional Networks for Action Recognition in Videos;Simonyan;Adv. Neural Inf. Process. Syst.,2014 3. Menze, M., and Geiger, A. (2015, January 7–12). Object scene flow for autonomous vehicles. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA. 4. Xu, R., Li, X., Zhou, B., and Loy, C.C. (2019, January 15–20). Deep Flow-Guided Video Inpainting. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA. 5. Yang, Y., Loquercio, A., Scaramuzza, D., and Soatto, S. (2019, January 15–20). Unsupervised Moving Object Detection via Contextual Information Separation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.
|
|