Author:
Javed Tahir,Doddapaneni Sumanth,Raman Abhigyan,Bhogale Kaushal Santosh,Ramesh Gowtham,Kunchukuttan Anoop,Kumar Pratyush,Khapra Mitesh M.
Abstract
Recent methods in speech and language technology pretrain very large models which are fine-tuned for specific tasks. However, the benefits of such large models are often limited to a few resource rich languages of the world. In this work, we make multiple contributions towards building ASR systems for low resource languages from the Indian subcontinent. First, we curate 17,000 hours of raw speech data for 40 Indian languages from a wide variety of domains including education, news, technology, and finance. Second, using this raw speech data we pretrain several variants of wav2vec style models for 40 Indian languages. Third, we analyze the pretrained models to find key features: codebook vectors of similar sounding phonemes are shared across languages, representations across layers are discriminative of the language family, and attention heads often pay attention within small local windows. Fourth, we fine-tune this model for downstream ASR for 9 languages and obtain state-of-the-art results on 3 public datasets, including on very low-resource languages such as Sinhala and Nepali. Our work establishes that multilingual pretraining is an effective strategy for building ASR systems for the linguistically diverse speakers of the Indian subcontinent.
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Cited by
16 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. A Self-Learning Framework for Large-Scale Conversational AI Systems;IEEE Computational Intelligence Magazine;2024-05
2. Multimodal Modeling for Spoken Language Identification;ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP);2024-04-14
3. Lightweight, Multi-Speaker, Multi-Lingual Indic Text-to-Speech;IEEE Open Journal of Signal Processing;2024
4. MASR: Multi-Label Aware Speech Representation;2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU);2023-12-16
5. Gated Multi Encoders and Multitask Objectives for Dialectal Speech Recognition in Indian Languages;2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU);2023-12-16