Multi-objective approach for designing ensemble of neural network classifiers with feature selection for emotion recognition problem

Reducing the dimensionality of datasets and configuring learning algorithms for solving particular practical tasks are the main problems in machine learning. In this work we propose the multi-objective optimization approach to feature selection and base learners hyper-parameter optimization. The effectiveness of the proposed multi-objective approach is compared to the single-objective approach. We chose emotion recognition problem by audio-visual data as a benchmark for comparing the two mentioned approaches. Also we chose neural network as a base learning algorithm for testing the proposed approach to parameter optimization. As a result of multi-objective optimization applied to parameter configuration we get the Pareto set of neural networks with optimal parameter values. In order to get the single output, the Pareto optimal neural networks were combined into an ensemble. We tried several ensemble model fusion techniques including voting, average class probabilities and meta-classification. According to the results, multi-objective optimization approach to feature selection provided an average 2.8 % better emotion classification rate on the given datasets than single-objective approach. Multi-objective approach is 5.4 % more effective compared to principal components analysis, and 13.9 % more effective compared to not using any dimensionality reduction at all. Multi-objective approach applied to neural networks parameter optimization provided on average 7.1 % higher classification rate than single-objective approach. The results suggest that the multi-objective optimization approach proposed in this article is more effective at solving considered emotion recognition problem.