Multi-objective based feature selection and neural networks ensemble method for solving emotion recognition problem

In this paper we apply multi-objective optimization approach to find a Pareto optimal ensemble of neural network classifiers, which is used for solving the emotion recognition problem. Pareto set of neural networks is found by optimizing two conflicting criteria: maximizing emotion classification rate and minimizing the number of neural network neurons. We implemented several ensemble fusion schemes - voting, averaging class probabilities and adding auxiliary meta-classification layer. The number of audio and video features extracted from raw video sequences for analysis is quite large, so we also applied multi-objective approach in order to find the optimal subset of features. The optimized criteria in this case are maximizing classification rate and minimizing the number of features. The multi-objective approach to neural network parameter optimization and to feature selection was compared to the classic single-objective optimization approach on several datasets. According to experimental results, multi-objective approach to neural net optimization provided on average 7.1 % higher emotion classification rate than single-objective optimization. Applying multi-objective approach to feature selection as well helped to improve the classification rate by 2.8 % compared to single-objective approach, by 5.4 % compared to using principal components analysis, and by 13.9 % compared to not using dimensionality reduction at all. Taking into account the obtained results, we suggest using multi-objective approach to machine learning algorithms optimization and feature selection in further research connected with emotion recognition problem and other complex classification tasks.