Novelty, confidence & errors in connectionist systems
Connectionist systems, or neural networks, have made a signiicant impact in the world of statistical pattern classiication, detection and prediction over the last few years. It is clear that the basic mechanisms involved with the method are sound. It is desirable, however, in all data analysis applications, to have estimates of the errors associated with the network's outputs. Whilst this is an established requirement for prediction and regression problems its eeects are subtle in the classiication case and are often ignored. Errors or uncertainty may be attributed to several causes within the network formalism; target noise and parameter uncertainty as well as, in certain cases, the variance of a committee of networks. We present in this paper a uniied, but simple, framework in which these errors may be evaluated. This includes assessment, even in the case of classiication, of the reduction in certainty in decisions made on`novel' inputs. This is of particular importance as if unseen data falls far outside the statistics of the training set then all input-output mappings are no longer valid and meaningless outputs are obtained.