Affiliation:
1. Cornell University and Peking University
2. Cornell University and McGill University
3. Cornell University
4. Cornell University, Ithaca, New York
Abstract
Facial expressions are highly informative for computers to understand and interpret a person's mental and physical activities. However, continuously tracking facial expressions, especially when the user is in motion, is challenging. This paper presents NeckFace, a wearable sensing technology that can continuously track the full facial expressions using a neck-piece embedded with infrared (IR) cameras. A customized deep learning pipeline called NeckNet based on Resnet34 is developed to learn the captured infrared (IR) images of the chin and face and output 52 parameters representing the facial expressions. We demonstrated NeckFace on two common neck-mounted form factors: a necklace and a neckband (e.g., neck-mounted headphones), which was evaluated in a user study with 13 participants. The study results showed that NeckFace worked well when the participants were sitting, walking, or after remounting the device. We discuss the challenges and opportunities of using NeckFace in real-world applications.
Publisher
Association for Computing Machinery (ACM)
Subject
Computer Networks and Communications,Hardware and Architecture,Human-Computer Interaction
Reference70 articles.
1. On-Body Sensing Solutions for Automatic Dietary Monitoring
2. Amonzon. [n.d.]. Musou USB Safety Tester USB Digital Power Meter Tester Multimeter Current and Voltage Monitor DC 5.1A 30V Amp Voltage Power Meter Test Speed of Chargers Cables Capacity of Power Banks Black. [EB/OL]. https://www.amazon.com/Musou-Digital-Multimeter-Chargers-Capacity/dp/B071214RD8 Accessed Oct 4 2020. Amonzon. [n.d.]. Musou USB Safety Tester USB Digital Power Meter Tester Multimeter Current and Voltage Monitor DC 5.1A 30V Amp Voltage Power Meter Test Speed of Chargers Cables Capacity of Power Banks Black. [EB/OL]. https://www.amazon.com/Musou-Digital-Multimeter-Chargers-Capacity/dp/B071214RD8 Accessed Oct 4 2020.
3. CanalSense
Cited by
30 articles.
订阅此论文施引文献
订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献
1. A Real-Time and Privacy-Preserving Facial Expression Recognition System Using an AI-Powered Microcontroller;Electronics;2024-07-16
2. EyeEcho: Continuous and Low-power Facial Expression Tracking on Glasses;Proceedings of the CHI Conference on Human Factors in Computing Systems;2024-05-11
3. UR2M: Uncertainty and Resource-Aware Event Detection on Microcontrollers;2024 IEEE International Conference on Pervasive Computing and Communications (PerCom);2024-03-11
4. Effective Facial Expression Recognition System Using Machine Learning;EAI Endorsed Transactions on Internet of Things;2024-03-11
5. UFace;Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies;2024-03-06