Dynamics of On-Line Gradient Descent Learning for Multilayer Neural Networks
We consider the problem of on-line gradient descent learning for general two-layer neural networks. An analytic solution is presented and used to investigate the role of the learning rate in controlling the evolution and convergence of the learning process.