Optimized Ensemble Machine Learning Approach for Emotion Detection from Thermal Images

Abstract

Emotions indicate the feelings of the individual which are linked with personal experiences, moods, and affective states. Detection of emotion can be helpful in many fields like maintaining a patient’s psychological well-being, surveillance, driver monitoring, etc. In this paper, an effective machine learning approach has been put forth for emotion detection where an ensemble of three out of five best-performing classifiers has been formed to enhance the classification accuracy. Two deep learning models (AlexNet and ResNet) have been optimally combined with [Formula: see text]-nearest neighbor (KNN). The optimal weights for ensemble weighted averaging of classifiers have been computed with aid of particle swarm optimization (PSO) and genetic algorithm (GA) optimization. The developed framework has been tested on two publicly available datasets. An overall accuracy of above 95% has been achieved on the testing set for both datasets. The best performance was obtained by training the classifiers with segmented images and combining them by using the weights obtained through PSO. The results depicted the efficiency of the optimized ensemble machine learning approach for all performance measures used in this study in comparison to the performance of individual classifiers and majority voting fusion.