Asymptotic Theory of Locally Conic Models and its Applications to Neural Networks
Multilayer neural networks have a problem of unidentiÞability in its parameterization. If a network has surplus hidden units to realize a target function, the parameters to give the function consist of a high dimensional subset. Many of usual statistical views fail in such cases. This paper discusses the likelihood ratio of the maximum likelihood estimation in unidentiÞable cases, using the framework of locally conic models. We derive a sufficient condition that the likelihood ratio has a larger order than usual Op(1). The exact order of the likelihood ratio of multilayer perceptrons is derived, and a new regularization scheme is proposed to overcome the strong overÞtting.