Abstract
ABSTRACTWe introduce the Bioacoustic Cocktail Party Problem Network (BioCPPNet), a lightweight, modular, and robust UNet-based machine learning architecture optimized for bioacoustic source separation across diverse biological taxa. Employing learnable or handcrafted encoders, BioCPPNet operates directly on the raw acoustic mixture waveform containing overlapping vocalizations and separates the input waveform into estimates corresponding to the sources in the mixture. Predictions are compared to the reference ground truth waveforms by searching over the space of (output, target) source order permutations, and we train using an objective function motivated by perceptual audio quality. We apply BioCPPNet to several species with unique vocal behavior, including macaques, bottlenose dolphins, and Egyptian fruit bats, and we evaluate reconstruction quality of separated waveforms using the scale-invariant signal-to-distortion ratio (SI-SDR) and downstream identity classification accuracy. We consider mixtures with two or three concurrent conspecific vocalizers, and we examine separation performance in open and closed speaker scenarios. To our knowledge, this paper redefines the state-of-the-art in end-to-end single-channel bioacoustic source separation in a permutation-invariant regime across a heterogeneous set of non-human species. This study serves as a major step toward the deployment of bioacoustic source separation systems for processing substantial volumes of previously unusable data containing overlapping bioacoustic signals.
Publisher
Cold Spring Harbor Laboratory
Reference59 articles.
1. The cocktail party problem: What is it? How can it be solved? And why should animal behaviorists study it?
2. Hershey, J. R. , Chen, Z. , Le Roux, J. & Watanabe, S. Deep clustering: Discriminative embeddings for segmentation and separation. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 31–35, DOI: https://doi.org/10.1109/ICASSP.2016.7471631 (2016).
3. Conv-TasNet: Surpassing ideal time–frequency magnitude masking for speech separation;IEEE/ACM Transactions on Audio, Speech, Lang. Process,2019
4. Chen, J. , Mao, Q. & Liu, D. Dual-path transformer network: Direct context-aware modeling for end-to-end monaural speech separation. Preprint at https://arxiv.org/abs/2007.13975 (2020).
5. Zeghidour, N. & Grangier, D. Wavesplit: End-to-end speech separation by speaker clustering. Preprint at https://arxiv.org/abs/2002.08933 (2020).