Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets-Reference-Cited by-同舟云学术

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets

Published:2022 Issue: Volume:10 Page:50-72
ISSN:2307-387X
Container-title:Transactions of the Association for Computational Linguistics
language:en
Short-container-title:

Author:

Kreutzer Julia¹²,Caswell Isaac³,Wang Lisa³⁴,Wahab Ahsan⁵⁶,van Esch Daan⁷,Ulzii-Orshikh Nasanbayar⁸,Tapo Allahsera⁹¹⁰,Subramani Nishant²¹¹,Sokolov Artem⁴,Sikasote Claytone¹²¹³,Setyawan Monang¹⁴,Sarin Supheakmungkol¹⁴,Samb Sokhar¹⁵¹⁶,Sagot Benoît¹⁷,Rivera Clara¹⁸,Rios Annette¹⁹,Papadimitriou Isabel²⁰,Osei Salomey²¹²²,Suarez Pedro Ortiz¹⁷²³,Orife Iroro²²⁴,Ogueji Kelechi²²⁵,Rubungo Andre Niyongabo²⁶²⁷,Nguyen Toan Q.²⁸,Müller Mathias¹⁹,Müller André¹⁹,Muhammad Shamsuddeen Hassan²⁹³⁰,Muhammad Nanda³⁰,Mnyakeni Ayanda³¹,Mirzakhalov Jamshidbek⁵⁶,Matangira Tapiwanashe³²,Leong Colin²,Lawson Nze¹⁴,Kudugunta Sneha³,Jernite Yacine²³³,Jenny Mathias¹⁹,Firat Orhan³⁵,Dossou Bonaventure F. P.³⁴³⁵,Dlamini Sakhile¹⁴,de Silva Nisansa³⁶,Çabuk Ballı Sakine¹⁹,Biderman Stella³⁷,Battisti Alessia¹⁹,Baruwa Ahmed²³⁸,Bapna Ankur³,Baljekar Pallavi¹,Azime Israel Abebe³⁹⁴⁰,Awokoya Ayodele²⁹⁴¹,Ataman Duygu¹⁹⁴²,Ahia Orevaoghene²⁴³,Ahia Oghenefego¹⁴,Agrawal Sweta⁴⁴,Adeyemi Mofetoluwa²⁹⁴⁵

Affiliation:

1. Google Research, Canada

2. Masakhane NLP, USA

3. Google Research, USA

4. Google Research, Germany

5. Turkic Interlingua

6. University of South Florida, USA

7. Google Research, The Netherlands

8. Haverford College, USA

9. Masakhane NLP, Mali

10. RobotsMali, Mali

11. Allen Institute for Artificial Intelligence, USA

12. Masakhane NLP, Zambia

13. University of Zambia, Zambia

14. Google, USA

15. Masakhane NLP, Senegal

16. AIMS-AMMI, Senegal

17. Inria, France

18. Google Research, UK

19. University of Zurich, Switzerland

20. Stanford University, USA

21. Masakhane NLP, Ghana

22. Kwame Nkrumah University of Science and Technology, Ghana

23. Sorbonne Université, France

24. Niger-Volta LTI, USA

25. University of Waterloo, Canada

26. Masakhane NLP, Spain

27. Universitat Politècnica de Catalunya, Spain

28. University of Notre Dame, USA

29. Masakhane NLP, Nigeria

30. Bayero University Kano, Nigeria

31. Google, South Africa

32. Google, Canada

33. Hugging Face, USA

34. Masakhane NLP, Germany

35. Jacobs University Bremen, Germany

36. University of Moratuwa, Sri Lanka

37. EleutherAI, USA

38. Obafemi Awolowo University, Nigeria

39. Masakhane NLP, Ethiopia

40. AIMS-AMMI, Ethiopia

41. University of Ibadan, Nigeria

42. Turkic Interlingua, Switzerland

43. Instadeep, Nigeria

44. University of Maryland, USA

45. Defence Space Administration Abuja, Nigeria

Abstract

AbstractWith the success of large-scale pre-training and multilingual modeling in Natural Language Processing (NLP), recent years have seen a proliferation of large, Web-mined text datasets covering hundreds of languages. We manually audit the quality of 205 language-specific corpora released with five major public datasets (CCAligned, ParaCrawl, WikiMatrix, OSCAR, mC4). Lower-resource corpora have systematic issues: At least 15 corpora have no usable text, and a significant fraction contains less than 50% sentences of acceptable quality. In addition, many are mislabeled or use nonstandard/ambiguous language codes. We demonstrate that these issues are easy to detect even for non-proficient speakers, and supplement the human audit with automatic analyses. Finally, we recommend techniques to evaluate and improve multilingual corpora and discuss potential risks that come with low-quality data releases.

Publisher

MIT Press

Subject

Artificial Intelligence,Computer Science Applications,Linguistics and Language,Human-Computer Interaction,Communication

Link

https://direct.mit.edu/tacl/article-pdf/doi/10.1162/tacl_a_00447/1986585/tacl_a_00447.pdf

Reference59 articles.

1. JW300: A wide-coverage parallel corpus for low-resource languages;Agić,2019

2. AraELECTRA: Pre-training text discriminators for Arabic language understanding;Antoun,2021

3. Massively multilingual neural machine translation in the wild: Findings and challenges;Arivazhagan;arXiv preprint arXiv:1907.05019,2019

4. Massively multilingual sentence embeddings for zero-shot cross-lingual transfer and beyond;Artetxe;Transactions of the Association for Computational Linguistics,2019

5. Domain adaptation via pseudo in-domain data selection;Axelrod,2011

Cited by 33 articles. 订阅此论文施引文献订阅此论文施引文献，注册后可以免费订阅5篇论文的施引文献，订阅后可以查看论文全部施引文献

1. Large models of what? Mistaking engineering achievements for human linguistic agency;Language Sciences;2024-11

2. A large-scale audit of dataset licensing and attribution in AI;Nature Machine Intelligence;2024-08-30

3. FinDeBERTaV2: Word-Segmentation-Free Pre-trained Language Model for Finance;Transactions of the Japanese Society for Artificial Intelligence;2024-07-01

4. Addressing the data gap: building a parallel corpus for Kashmiri language;International Journal of Information Technology;2024-06-21

5. Demystifying Data Management for Large Language Models;Companion of the 2024 International Conference on Management of Data;2024-06-09