Neural network with manifold recurrent structure

The structure and learning procedure is analyzed for neural network (NN) with manifold feedback structure grouped in several layers. The difference of this NN compared with recurrent networks is that temporal relations are provided by means of neurons arranged in three feedback layers that enrich the representation capabilities of NN. Feedback layers provide local and global recurrences via nonlinear processing elements. In feedback layers weighed sums of the delayed outputs of hidden and output layers are passed through certain activation functions and applied to the feedforward neurons via adjustable weights. The learning procedures for NN (including real time learning) are given that are based on back propagation through time algorithm. The "adjoint" model for NN with manifold recurrent structure is given that diminish the computation complexity of constructing NN learning algorithm.