Sequential Monte Carlo methods for optimisation of neural network models
We discuss a novel strategy for training neural networks using sequential Monte Carlo algorithms and propose a new hybrid gradient descent/sampling importance resampling algorithm (HySIR). In terms of both computational time and accuracy, the hybrid SIR is a clear improvement over conventional sequential Monte Carlo techniques. The new algorithm may be viewed as a global optimisation strategy, which allows us to learn the probability distributions of the network weights and outputs in a sequential framework. It is well suited to applications involving on-line, nonlinear and non-Gaussian signal processing. We show how the new algorithm outperforms extended Kalman lter training on several problems. In particular, we address the problem of pricing option contracts, traded in nancial markets. In this context, we are able to estimate the one-step-ahead probability density functions of the options prices. i