A convolutional neural network-based framework for analysis and assessment of non-linguistic sound classification and enhancement for normal hearing and cochlear implant listeners

Abstract

Naturalistic sounds encode salient acoustic content that provides situational context or subject/system properties essential for acoustic awareness, autonomy, safety, and improved quality of life for individuals with sensorineural hearing loss. Cochlear implants (CIs) are an assistive hearing device that restores auditory function in hearing impaired individuals. Most CI research advancements have focused on improving speech recognition in noisy, reverberant, or time-varying diverse environments. Relatively few studies have explored non-linguistic sound (NLS) perception among CIs, and those that have carried out such studies generally reported poor perception, suggesting a clear deficit in current CI sound processing systems. In this study, a convolutional neural network (CNN)-based NLS classification model is used as a framework to compare unprocessed and CI-simulated NLS classification and evaluate NLS perception targeted algorithms among CI listeners. Additionally, a NLS enhancement algorithm that focuses on improving identifiability and perception among CI listeners is proposed. The proposed NLS enhancement algorithm is evaluated based on identifiability performance using the CI-simulated NLS classification model. The proposed NLS classification framework was able to achieve near human-level performance with no significant effect of classification modality (model vs human subject) and achieved mean classification scores of 85.86% for NH ( p = 0.3758) and 65.25% for CI ( p = 0.1725). Among the four different feature-based methods of the proposed NLS enhancement algorithm, the “harmonicity”-based one achieved highest mean classification accuracy of 63.75%, when compared to baseline, and demonstrated significant improvement in performance ( p = 0.0403). The resulting proposed comparative NLS classification framework contributes toward (i) advancement of NLS recognition studies, (ii) mitigation of CI user recruitment constraints and listener evaluation with NH listeners, (iii) development of a community shared testbed for comparative NLS studies, and (iv) advancement of NLS enhancement studies (identifiability and perceptual factors) among CI listeners.