On the Use of Pitch-Based Features for Detecting Simultaneous Fear Emotion and Deception Behavior From Speech

Abstract

In the last years, endowing the machine with emotional and behavioral intelligence has been one of the most challenging issues in the human–computer interaction area. In this context, this paper investigates the detection of simultaneous fear emotion and deception behavior in speech. To do so, a set of 72 pitch-based features has been investigated first to recognize fear and deception separately. Then, different feature selection techniques have been used in order to select the most relevant ones that best discriminate between fear/nonfear and deception/nondeception classes. Next, a decision-level fusion approach based on the belief theory has been proposed to infer whether fear and deception are detected simultaneously. Simulation results carried on databases dealing with fear/nonfear emotions and deception/truth behaviors have shown classification results reaching 83.33% and 72.45% as accuracy rates for fear and deception classifiers, respectively. The proposed fusion approach has revealed a correspondence between fear emotion and deception behavior in speech modality.