I Spy You

Author:

Zhang Shijia1ORCID,Liu Yilin1ORCID,Gowda Mahanth1ORCID

Affiliation:

1. The Pennsylvania State University, USA

Abstract

This paper presents iSpyU, a system that shows the feasibility of recognition of natural speech content played on a phone during conference calls (Skype, Zoom, etc) using a fusion of motion sensors such as accelerometer and gyroscope. While microphones require permissions from the user to be accessible by an app developer, the motion sensors are zero-permission sensors, thus accessible by a developer without alerting the user. This allows a malicious app to potentially eavesdrop on sensitive speech content played by the user's phone. In designing the attack, iSpyU tackles a number of technical challenges including: (i) Low sampling rate of motion sensors (500 Hz in comparison to 44 kHz for a microphone). (ii) Lack of availability of large-scale training datasets to train models for Automatic Speech Recognition (ASR) with motion sensors. iSpyU systematically addresses these challenges by a combination of techniques in synthetic training data generation, ASR modeling, and domain adaptation. Extensive measurement studies on modern smartphones show a word level accuracy of 53.3 - 59.9% over a dictionary of 2000-10000 words, and a character level accuracy of 70.0 - 74.8%. We believe such levels of accuracy poses a significant threat when viewed from a privacy perspective.

Funder

National Science Foundation

Publisher

Association for Computing Machinery (ACM)

Subject

Computer Networks and Communications,Hardware and Architecture,Human-Computer Interaction

Reference96 articles.

1. Dario Amodei , Sundaram Ananthanarayanan , Rishita Anubhai , Jingliang Bai , Eric Battenberg , Carl Case , Jared Casper , Bryan Catanzaro , Qiang Cheng , Guoliang Chen , 2016 . Deep speech 2: End-to-end speech recognition in english and mandarin . In International conference on machine learning. PMLR, 173--182 . Dario Amodei, Sundaram Ananthanarayanan, Rishita Anubhai, Jingliang Bai, Eric Battenberg, Carl Case, Jared Casper, Bryan Catanzaro, Qiang Cheng, Guoliang Chen, et al. 2016. Deep speech 2: End-to-end speech recognition in english and mandarin. In International conference on machine learning. PMLR, 173--182.

2. Speechless: Analyzing the Threat to Speech Privacy from Smartphone Motion Sensors

3. S Abhishek Anand , Chen Wang , Jian Liu , Nitesh Saxena , and Yingying Chen . 2019 . Spearphone: A speech privacy exploit via accelerometer-sensed reverberations from smartphone loudspeakers. arXiv preprint arXiv:1907.05972 (2019). S Abhishek Anand, Chen Wang, Jian Liu, Nitesh Saxena, and Yingying Chen. 2019. Spearphone: A speech privacy exploit via accelerometer-sensed reverberations from smartphone loudspeakers. arXiv preprint arXiv:1907.05972 (2019).

4. Android Sensors 2022. Sensors Overview. https://developer.android.com/guide/topics/sensors/sensors_overview. Android Sensors 2022. Sensors Overview. https://developer.android.com/guide/topics/sensors/sensors_overview.

5. Safaa Azzakhnini and Ralf C Staudemeyer . 2020. Extracting speech from motion-sensitive sensors . In Data Privacy Management, Cryptocurrencies and Blockchain Technology . Springer , 145--160. Safaa Azzakhnini and Ralf C Staudemeyer. 2020. Extracting speech from motion-sensitive sensors. In Data Privacy Management, Cryptocurrencies and Blockchain Technology. Springer, 145--160.

Cited by 3 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

1. Stealthy and Practical Multi-modal Attacks on Mixed Reality Tracking;2024 IEEE International Conference on Artificial Intelligence and eXtended and Virtual Reality (AIxVR);2024-01-17

2. Practical Earphone Eavesdropping with Built-in Motion Sensors;2023 IEEE 29th International Conference on Parallel and Distributed Systems (ICPADS);2023-12-17

3. I Am an Earphone and I Can Hear My User’s Face: Facial Landmark Tracking Using Smart Earphones;ACM Transactions on Internet of Things;2023-12-16

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3