Reconstructed Prototype Network Combined with CDC-TAGCN for Few-Shot Action Recognition

Abstract

Research on few-shot action recognition has received widespread attention recently. However, there are some blind spots in the current research: (1) The prevailing practice in many models is to assign uniform weights to all samples; nevertheless, such an approach may yield detrimental consequences for the model in the presence of high-noise samples. (2) Samples with similar features but different classes make it difficult for the model to be distinguished. (3) Skeleton data harbors rich temporal features, but most encoders face challenges in effectively extracting them. In response to these challenges, this study introduces a reconstructed prototype network (RC-PN) based on a prototype network framework and a novel spatiotemporal encoder. The RC-PN comprises two enhanced modules: Sample coefficient reconstruction (SCR) and a reconstruction loss function (LRC). SCR leverages cosine similarity between samples to reassign sample weights, thereby generating prototypes robust to noise interference and more adept at conveying conceptual essence. Simultaneously, the introduction of LRC enhances the feature similarity among samples of the same class while increasing feature distinctiveness between different classes. In the encoder aspect, this study introduces a novel spatiotemporal convolutional encoder called CDC-TAGCN. The temporal convolution operator is redefined in CDC-TAGCN. The vanilla temporal convolution operator can only capture the surface-level characteristics of action samples. Drawing inspiration from differential convolution (CDC), this research enhances TCN to CDC-TGCN. CDC-TGCN allows for the fusion of discrepant features from action samples into the features extracted by the vanilla convolutional operator. Extensive feasibility and ablation experiments are performed on the skeleton action dataset NTU-RGB + D 120 and Kinetics and compared with recent research.