Hybrid algorithm for convolutional neural network learning

The problems of image pattern recognition are nowadays solved in many areas - beginning with satellite analysis of the Earth surface all the way to human face analysis in the human-machine interaction systems. One of the most successful algorithms of image analysis and recognition is the convolutional neural network. The back propagation algorithm used for training of these networks belongs to the class of gradient algorithms; therefore it often converges to local minima. In this paper we propose a hybrid algorithm for convolutional network training, which is aimed at searching for global optimum, and consisting of two stages. At the first stage a genetic algorithm is used to search for the subdomain of the search space that includes the global optimum, whereas at the second stage a back propagation algorithm is used to find the actual global optimum. Genetic algorithm includes an internal procedure that maintains diversity of the solutions population, which allows examining the search space more thoroughly and finding various successful convolutional network configurations. The solution found by the genetic algorithm is used at the second stage as an initial approximation of network weights. After that the network is trained using a back propagation algorithm. The developed hybrid algorithm has been tested on the emotion recognition problem; it has been compared to a traditional back propagation algorithm. A comparison was made on the classification accuracy, as well as the F-measure for the emotion recognition problem in two formulations - speaker-dependent classification, and speaker-independent classification. Hybrid algorithm has yielded a better performance according to both criteria, and in both problem formulations compared to the standard algorithm used for convolutional neural network training.