Assessing thermal imagery integration into object detection methods on air-based collection platforms

Abstract

Abstract Object detection models commonly focus on utilizing the visible spectrum via Red-Green-Blue (RGB) imagery. Due to various limitations with this approach in low visibility settings, there is growing interest in fusing RGB with thermal long wave infrared (LWIR) (7.5–13.5 µm) images to increase object detection performance. However, we still lack baseline performance metrics evaluating RGB, LWIR and RGB-LWIR fused object detection machine learning models, especially from air-based platforms. This study undertakes such an evaluation finding that a blended RGB-LWIR model generally exhibits superior performance compared to traditional RGB or LWIR approaches. For example, an RGB-LWIR blend only performed 1–5% behind the RGB approach in predictive power across various altitudes and periods of clear visibility. Yet, RGB fusion with a thermal signature overlayed provides edge redundancy and edge emphasis, both which are vital in supporting edge detection machine learning algorithms. This approach has the ability to improve object detection performance for a range of use cases in industrial, consumer, government, and military applications. Finally, this research additionally contributes a novel open labeled training dataset of 6,300 images for RGB, LWIR, and RGB-LWIR fused imagery, collected from air-based platforms, enabling further multispectral machine-driven object detection research.