Differentiating Functions of the Jacobianwith Respect to the Weights
For many problems, the correct behavior of a model depends not only on 
its input-output mapping but also on properties of its Jacobian matrix, the 
matrix of partial derivatives of the model's outputs with respect to its inputs. 
We introduce the J-prop algorithm, an efficient general method for 
computing the exact partial derivatives of a variety of simple functions of 
the Jacobian of a model with respect to its free parameters. The algorithm 
applies to any parametrized feedforward model, including nonlinear regression, 
multilayer perceptrons, and radial basis function networks.