Affiliation:
1. Tsinghua University, Beijing, China
2. Alibaba DAMO Academy, Hangzhou, China
Abstract
Millimeter wave (mmWave) based sensing is a significant technique that enables innovative smart applications, e.g., voice recognition. The existing works in this area require direct sensing of the human's near-throat region and consequently have limited applicability in non-line-of-sight (NLoS) scenarios. This paper proposes AmbiEar, the first mmWave based voice recognition approach applicable in NLoS scenarios. AmbiEar is based on the insight that the human's voice causes correlated vibrations of the surrounding objects, regardless of the human's position and posture. Therefore, AmbiEar regards the surrounding objects as ears that can perceive sound and realizes indirect sensing of the human's voice by sensing the vibration of the surrounding objects. By incorporating the designs like common component extraction, signal superimposition, and encoder-decoder network, AmbiEar tackles the challenges induced by low-SNR and distorted signals. We implement AmbiEar on a commercial mmWave radar and evaluate its performance under different settings. The experimental results show that AmbiEar has a word recognition accuracy of 87.21% in NLoS scenarios and reduces the recognition error by 35.1%, compared to the direct sensing approach.
Funder
NSFC
The Guoqiang Institute, Tsinghua University
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Networks and Communications,Hardware and Architecture,Human-Computer Interaction
Reference54 articles.
1. Automatic Speaker Recognition System in Adverse Conditions — Implication of Noise and Reverberation on System Performance
2. A generalized MVDR spectrum
3. Google Brain. 2017. TensorFlow Speech Recognition Challenge. https://www.kaggle.com/c/tensorflow-speech-recognition-challenge. Google Brain. 2017. TensorFlow Speech Recognition Challenge. https://www.kaggle.com/c/tensorflow-speech-recognition-challenge.
4. ThermoWave
5. Yann N Dauphin , Angela Fan , Michael Auli , and David Grangier . 2017 . Language modeling with gated convolutional networks . In International conference on machine learning. PMLR, 933--941 . Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier. 2017. Language modeling with gated convolutional networks. In International conference on machine learning. PMLR, 933--941.
Cited by
7 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献
1. OCro: Open-Set Cross-Domain Human Activity Recognition Based on Radio Frequency;IEEE Internet of Things Journal;2023-12-15
2. CubeSense++: Smart Environment Sensing with Interaction-Powered Corner Reflector Mechanisms;Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology;2023-10-29
3. Headar;Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies;2023-09-27
4. mmHawkeye: Passive UAV Detection with a COTS mmWave Radar;2023 20th Annual IEEE International Conference on Sensing, Communication, and Networking (SECON);2023-09-11
5. Contactless Material Identification with Millimeter Wave Vibrometry;Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services;2023-06-18