A Novel Hierarchical Framework for Measuring the Complexity and Irregularity of Multimodal Speech Signals and Its Application in the Assessment of Speech Impairment in Amyotrophic Lateral Sclerosis

Abstract

Purpose The purposes of this study are to develop a novel multimodal framework for measuring variability at the muscular, kinematic, and acoustic levels of the motor speech hierarchy and evaluate the utility of this framework in detecting speech impairment in amyotrophic lateral sclerosis (ALS). Method The myoelectric activities of three bilateral jaw muscle pairs (masseter, anterior temporalis, and anterior belly of digastric), jaw kinematics, and speech acoustics were recorded in 13 individuals with ALS and 10 neurologically healthy controls during sentence reading. Thirteen novel measures (six muscular, three kinematic, four acoustic), which characterized two different but interrelated aspects of variability—complexity and irregularity—were derived using linear and nonlinear methods. Exploratory factor analysis was applied to identify the latent factors underlying these measures. Based on the latent factors, three supervised classifiers—support vector machine (SVM), random forest (RF), and logistic regression (Logit)—were used to differentiate between the speech samples for patients and controls. Results Four interpretable latent factors were identified, representing the complexity of jaw kinematics, the irregularity of jaw antagonists functioning, the irregularity of jaw agonists functioning, and the irregularity of subband acoustic signals, respectively. Based on these latent factors, the speech samples for patients and controls were classified with high accuracy (> 96% for SVM and RF; 88.64% for Logit), outperforming the unimodal measures. Two factors showed significant between-groups differences, as characterized by decreased complexity of jaw kinematics and increased irregularity of jaw antagonists functioning in patients versus controls. Conclusions Decreased complexity of jaw kinematics presumably reflects impaired fine control of jaw movement, while increased irregularity of jaw antagonists functioning could be attributed to reduced synchronization of motor unit firing in ALS. The findings provide preliminary evidence for the utility of the multimodal framework as a novel quantitative assessment tool for detecting speech impairment in ALS and (potentially) in other neuromotor disorders.