Exploring Collection of Sign Language Videos through Crowdsourcing

Author:

Bragg Danielle1,Glasser Abraham2,Minakov Fyodor1,Caselli Naomi3,Thies William1

Affiliation:

1. Microsoft Research, Cambridge, MA, USA

2. Rochester Institute of Technology, Rochester, NY, USA

3. Boston University, Boston, MA, USA

Abstract

Inadequate sign language data currently impedes advancement of sign language ML and AI. Training on existing datasets results in limited models due to small size, and lack of diverse signers in real-world settings. Complex labeling problems in particular often limit scale. In this work, we explore the potential for crowdsourcing to help overcome these barriers. To do this, we ran a user study with exploratory crowdsourcing tasks designed to support scalability: 1) to record videos of specific content -- thereby enabling automatic, scalable labeling -- and 2) to perform quality control checks for execution consistency -- further reducing post-processing requirements. We also provided workers with a searchable view of the crowdsourced dataset, to boost engagement and transparency and align with Deaf community values. Our user study included 29 participants using our exploratory tasks to record 1906 videos and perform 2331 quality control checks. Our results suggest that a crowd of signers may be able to generate high-quality recordings and perform reliable quality control, and that the signing community values visibility into the resulting dataset.

Funder

NSF

Publisher

Association for Computing Machinery (ACM)

Subject

Computer Networks and Communications,Human-Computer Interaction,Social Sciences (miscellaneous)

Reference63 articles.

1. Amazon. 2005. Amazon Mechanical Turk. https://www.mturk.com/ Accessed 2021-08--27. Amazon. 2005. Amazon Mechanical Turk. https://www.mturk.com/ Accessed 2021-08--27.

2. The American Sign Language Lexicon Video Dataset

3. VizWiz

4. Gauging Receptiveness to Social Microvolunteering

5. Danielle Bragg , Naomi Caselli , Julie A. Hochgesang , Matt Huenerfauth , Leah Katz-Hernandez , Oscar Koller , Raja Kushalnagar , Christian Vogler , and Richard E. Ladner . 2021b . The FATE Landscape of Sign Language AI Datasets: An Interdisciplinary Perspective. TACCESS 2021 ( 2021 ). Danielle Bragg, Naomi Caselli, Julie A. Hochgesang, Matt Huenerfauth, Leah Katz-Hernandez, Oscar Koller, Raja Kushalnagar, Christian Vogler, and Richard E. Ladner. 2021b. The FATE Landscape of Sign Language AI Datasets: An Interdisciplinary Perspective. TACCESS 2021 (2021).

Cited by 5 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

1. Interactive learning of natural sign language with radar;IET Radar, Sonar & Navigation;2024-05-10

2. Machine Learning Technology to Recognize American Sign Language Alphabet;2023 IEEE 20th International Conference on Smart Communities: Improving Quality of Life using AI, Robotics and IoT (HONET);2023-12-04

3. Alabib-65: A Realistic Dataset for Algerian Sign Language Recognition;ACM Transactions on Asian and Low-Resource Language Information Processing;2023-06-17

4. Contributing to Accessibility Datasets: Reflections on Sharing Study Data by Blind People;Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems;2023-04-19

5. Community-Driven Information Accessibility: Online Sign Language Content Creation within d/Deaf Communities;Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems;2023-04-19

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3