Pre-rotation Only at Inference-Stage: A Way to Rotation Invariance of Convolutional Neural Networks

Abstract

AbstractThe popular convolutional neural networks (CNN) require data augmentation to achieve rotation invariance. We propose an alternative mechanism, Pre-Rotation Only at Inference stage (PROAI), to make CNN rotation invariant. The overall idea is to learn how the human brain observe images. At the training stage, PROAI trains a CNN with a small number using images only at one orientation. At the inference stage, PROAI introduces a pre-rotation operation to rotate each test image into its all-possible orientations and calculate classification scores using the trained CNN with a small number of parameters. The maximum of these classification scores is able to simultaneously estimate both the category and the orientation of each test image. The specific benefits of PROAI have been experimented on rotated image recognition tasks. The results shows that PROAI improves both the classification and orientation estimation performance while greatly reduced the numbers of parameters and the training time. Codes and datasets are publicly available at https://github.com/automlresearch/FRPRF.