Aberration aware feature detection and description

Abstract

Feature detection and description are crucial for image matching, as better performance at this stage leads to more accurate matching results, which is essential for subsequent vision-based tasks. However, images captured by different optical systems may suffer from various optical aberrations, especially in off-axis field and out-of-depth-of-field regions, making it challenging for models to extract consistent feature locations and descriptors. In this paper, we propose what we believe to be a novel method for training feature detection and description networks by incorporating optical system aberrations modeled by point spread function(PSF). We introduce PSF augmentation and homographic PSF adaptation, which customize the training of feature detection and description models for specific optical systems using general unlabeled image datasets. Experimental results demonstrate that our method significantly improves the performance of feature detection and description in images captured by given cameras.