Affiliation:
1. Bosch Global Software Technologies Pvt Ltd , Bangalore , India
2. Bamboozoology LLC , San Francisco , CA , USA
3. Department of Cognitive Science , Case Western Reserve University , Cleveland , OH , USA
Abstract
Abstract
The International Distributed Little Red Hen Lab, usually called “Red Hen Lab” or just “Red Hen”, is dedicated to research into multimodal communication. In this article, we introduce the Red Hen Audio Tagger (RHAT), a novel, publicly available open source platform developed by Red Hen Lab. RHAT employs deep learning models to tag audio elements frame by frame, generating metadata tags that can be utilized in various data formats for analysis. RHAT seamlessly integrates with widely used linguistic research tools like ELAN: the researcher can use RHAT to tag audio content automatically and display those tags alongside other ELAN annotation tiers. RHAT additionally complements existing Red Hen pipelines devoted to natural language processing, speech-to-text processing, body pose analysis, optical character recognition, named entity recognition, computer vision, semantic frame recognition, and so on. These cooperating Red Hen pipelines are research tools to advance the science of multimodal communication.
Reference12 articles.
1. Brugman, Hennie & Albert Russel. 2004. Annotating multimedia/multi-modal resources with ELAN. In Maria, Teresa Lino, Maria, Francisca Xavier, Fátima, Ferreira, Rute, Costa & Raquel, Silva (eds.), Proceedings of the fourth international conference on language resources and evaluation. Lisbon, Portugal: European Language Resources Association.
2. Dolan, Stephen. 2018. JQ, A lightweight and flexible command-line JSON processor, version jq1.6 [Computer program]. Available at: https://stedolan.github.io/jq.
3. Ford, Logan, Hao Tang, François Grondin & James Glass. 2019. A deep residual network for large-scale acoustic scene analysis. Interspeech 2019. 2568–2572. https://doi.org/10.21437/Interspeech.2019-2731.
4. Gemmeke, Jort F., Daniel P. W. Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, R. Channing Moore, Manoj Plakal & Marvin Ritter. 2017. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE international conference on acoustics, speech & signal processing (ICASSP), 776–780. IEEE.
5. Gong, Yuan, Yu-An Chung & James Glass. 2021. AST: Audio spectrogram transformer. arXiv. https://doi.org/10.48550/arXiv.2104.01778.