NoteWordy: Investigating Touch and Speech Input on Smartphones for Personal Data Capture-Reference-Cited by-同舟云学术

NoteWordy: Investigating Touch and Speech Input on Smartphones for Personal Data Capture

Published:2022-11-14 Issue:ISS Volume:6 Page:568-591
ISSN:2573-0142
Container-title:Proceedings of the ACM on Human-Computer Interaction
language:en
Short-container-title:Proc. ACM Hum.-Comput. Interact.

Author:

Luo Yuhan¹^ORCID,Lee Bongshin²^ORCID,Kim Young-Ho³^ORCID,Choe Eun Kyoung⁴^ORCID

Affiliation:

1. City University of Hong Kong, Hong Kong, China

2. Microsoft Research, Redmond, USA

3. University of Maryland, College Park, USA / NAVER AI Lab, Seongnam, South Korea

4. University of Maryland, College Park, USA

Abstract

Speech as a natural and low-burden input modality has great potential to support personal data capture. However, little is known about how people use speech input, together with traditional touch input, to capture different types of data in self-tracking contexts. In this work, we designed and developed NoteWordy, a multimodal self-tracking application integrating touch and speech input, and deployed it in the context of productivity tracking for two weeks (N = 17). Our participants used the two input modalities differently, depending on the data type as well as personal preferences, error tolerance for speech recognition issues, and social surroundings. Additionally, we found speech input reduced participants' diary entry time and enhanced the data richness of the free-form text. Drawing from the findings, we discuss opportunities for supporting efficient personal data capture with multimodal input and implications for improving the user experience with natural language input to capture various self-tracking data.

Funder

National Science Foundation

Publisher

Association for Computing Machinery (ACM)

Subject

Computer Networks and Communications,Human-Computer Interaction,Social Sciences (miscellaneous)

Link

https://dl.acm.org/doi/pdf/10.1145/3567734

Reference53 articles.

1. Google Cloud-Send a recognition request with speech adaptation. https://cloud.google.com/speech-to-text/docs/ context-strength. Accessed: 2022-09-30. Google Cloud-Send a recognition request with speech adaptation. https://cloud.google.com/speech-to-text/docs/ context-strength. Accessed: 2022-09-30.

2. Kotlin. https://kotlinlang.org/. Accessed: 2022-09-30. Kotlin. https://kotlinlang.org/. Accessed: 2022-09-30.

3. Microsoft Cognitive Service-Prepare data for Custom Speech. https://docs.microsoft.com/en-us/azure/cognitiveservices/speech-service/ how-to-custom-speech-test-and-train. Accessed: 2022-09-30. Microsoft Cognitive Service-Prepare data for Custom Speech. https://docs.microsoft.com/en-us/azure/cognitiveservices/speech-service/ how-to-custom-speech-test-and-train. Accessed: 2022-09-30.

4. Microsoft Cognitive Service-Speech to Text. https://azure.microsoft.com/en-us/services/cognitive-services/speechto-text/. Accessed: 2022-09-30. Microsoft Cognitive Service-Speech to Text. https://azure.microsoft.com/en-us/services/cognitive-services/speechto-text/. Accessed: 2022-09-30.

5. Natty. http://natty.joestelmach.com/. Accessed: 2022-09-30. Natty. http://natty.joestelmach.com/. Accessed: 2022-09-30.

Cited by 4 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Improving Error Correction and Text Editing Using Voice and Mouse Multimodal Interface;International Journal of Human–Computer Interaction;2024-05-22

2. Fragmented Moments, Balanced Choices: How Do People Make Use of Their Waiting Time?;Proceedings of the CHI Conference on Human Factors in Computing Systems;2024-05-11

3. StayFocused: Examining the Effects of Reflective Prompts and Chatbot Support on Compulsive Smartphone Use;Proceedings of the CHI Conference on Human Factors in Computing Systems;2024-05-11

4. Emotion Embodied: Unveiling the Expressive Potential of Single-Hand Gestures;Proceedings of the CHI Conference on Human Factors in Computing Systems;2024-05-11