General spiking neural network framework for the learning trajectory from a noisy mmWave radar

Abstract

Abstract Emerging usages for millimeter wave (mmWave) radar have drawn extensive attention and inspired the exploration of learning mmWave radar data. To be effective, instead of using conventional approaches, recent works have employed modern neural network models to process mmWave radar data. However, due to some inevitable obstacles, e.g., noise and sparsity issues in data, the existing approaches are generally customized for specific scenarios. In this paper, we propose a general neuromorphic framework, termed mm-SNN, to process mmWave radar data with spiking neural networks (SNNs), leveraging the intrinsic advantages of SNNs in processing noisy and sparse data. Specifically, we first present the overall design of mm-SNN, which is adaptive and easily expanded for multi-sensor systems. Second, we introduce general and straightforward attention-based improvements into the mm-SNN to enhance the data representation, helping promote performance. Moreover, we conduct explorative experiments to certify the robustness and effectiveness of the mm-SNN. To the best of our knowledge, mm-SNN is the first SNN-based framework that processes mmWave radar data without using extra modules to alleviate the noise and sparsity issues, and at the same time, achieve considerable performance in the task of trajectory estimation.