UltraCLR: Contrastive Representation Learning Framework for Ultrasound-based Sensing

Author:

Wang Xun1,Yang Zhizheng1,Wang Wei1,Dai Haipeng1,Shi ShuYu1,Gu Qing1

Affiliation:

1. State Key Lab. for Novel Software Technology, Nanjing University, China

Abstract

We propose UltraCLR, a new contrastive learning framework that fuses dual modulation ultrasonic sensing signals to enhance gesture representation. Most existing ultrasound-based gesture recognition tasks rely on a large amount of manually labeled samples to learn task-specific representations via end-to-end training. However, they cannot exploit unlabeled continuous gesture signals that are easy to collect. Inspired by recent self-supervised learning techniques, UltraCLR aims to autonomously learn a ubiquitous gesture signal representation that can benefit all tasks from low-cost unlabeled signals. We use the STFT heatmap as a secondary input and leverage the contrastive learning framework to improve the high-quality Channel Impulsive Response (CIR) heatmap input representations. The learned representations can better represent the spatial-position information and intermediate states of gesture movement. With the representation learned by UltraCLR, we can greatly reduce the complexity of downstream gesture recognition tasks so that they can be completed using a simple classifier trained with a small training set and a lower computational cost. Our experimental results show that UltraCLR outperforms state-of-the-art gesture recognition systems with only a few labeled samples, and achieves more than 85% reduction in computational complexity and over 9x improvement in inference speed.

Publisher

Association for Computing Machinery (ACM)

Subject

Computer Networks and Communications

Reference47 articles.

1. IMU2Doppler: Cross-Modal Domain Adaptation for Doppler-Based Activity Recognition Using IMU Data;Bhalla Sejal;Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies,2022

2. Romil Bhardwaj , Zhengxu Xia , Ganesh Ananthanarayanan , Junchen Jiang , Yuanchao Shu , Nikolaos Karianakis , Kevin Hsieh , Paramvir Bahl , and Ion Stoica . 2022 . Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers . In Proceedings of USENIX NSDI. 119–135 . Romil Bhardwaj, Zhengxu Xia, Ganesh Ananthanarayanan, Junchen Jiang, Yuanchao Shu, Nikolaos Karianakis, Kevin Hsieh, Paramvir Bahl, and Ion Stoica. 2022. Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers. In Proceedings of USENIX NSDI. 119–135.

3. Mathilde Caron , Ishan Misra , Julien Mairal , Priya Goyal , Piotr Bojanowski , and Armand Joulin . 2020 . Unsupervised Learning of Visual Features by Contrasting Cluster Assignments . In Proceedings of NeurlPS. 9912–9924 . Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin. 2020. Unsupervised Learning of Visual Features by Contrasting Cluster Assignments. In Proceedings of NeurlPS. 9912–9924.

4. A Systematic Study of Unsupervised Domain Adaptation for Robust Human-Activity Recognition

5. Ting Chen , Simon Kornblith , Mohammad Norouzi , and Geoffrey Hinton . 2020 . A Simple Framework for Contrastive Learning of Visual Representations . In Proceedings of ICML. 1597–1607 . Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A Simple Framework for Contrastive Learning of Visual Representations. In Proceedings of ICML. 1597–1607.

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3