Optimization of training time of neural networks with adaptive learning rate parameters

The paper analyzes the classical gradient descent method and suggests a method for dynamically changing the learning step based on the calculated parameters τ and p. The main focus is on an algorithm that allows calculating the optimal values of the parameters τ and p to minimize the training time. The experiments demonstrate how changes in these parameters affect the learning rate for various neural network topologies and activation functions. The simulation results show that the correct choice of τ and p can significantly reduce the time required for training neural networks with a fixed structure. Using these parameters allows to improve the learning process, preventing getting stuck in local minima and ensuring a balance between the learning rate and the accuracy of the result. Research has demonstrated the effectiveness of an adaptive approach for various neural network topologies and activation functions. The presented graphs and numerical calculations show the dependence of the average learning rate on the selected parameters.