Low‐resolution activity recognition using super‐resolution and model ensemble networks

Abstract

AbstractIn real‐world video super‐resolution, the complexity and diversity of degradations pose substantial challenges during both training and inference. Videos captured in real‐world settings often depict activities at varying resolutions. Typically, these activities are filmed from a distance that reduces the resolution of imagery, which thus lacks discriminative features. To address this problem, we introduce an activity recognition solution. First, a unique integration of data transformation and attention‐based average discriminator are employed for super‐resolution feature augmentation. This approach mitigates the lack of discriminative cues in low‐resolution videos. Subsequently, high‐resolution features extracted from the recovered data are directly fed into a model ensemble for activity recognition. We evaluate the resulting method on the TinyVIRAT‐v2 and HMDB51 datasets, achieving improved visual quality by leveraging the super‐resolution and model ensemble strategy. The proposed method enhances the quality of textures and boosts activity recognition in low‐resolution videos.