Vc Dimension Further Information 3.3 Optimization Algorithms 3 Learning from Data
A measure of the complexity of a model. Knowledge of the VC dimension permits an estimate to be made of the diierence between performance on the training set and performance on a test set. 29 Deening terms Classiication A learning problem in which the goal is to assign input vectors to one of a number of (usually mutually exclusive) classes. Boltzmann machine An undirected network of discrete valued random variables, where an energy function is associated with each of the links, and for which a probability distribution is deened by the Boltzmann distribution. Cost function A function of the adaptive parameters of a model whose minimum is used to deene suitable values for those parameters. It may consist of a likelihood function and additional terms. Decision tree A network that performs a sequence of classiicatory decisions on an input vector and produces an output vector that is conditional on the outcome of the decision sequence. Density estimation The problem of modeling a probability distribution from a nite set of examples drawn from that distribution. Discriminant function A function of the input vector which can be used to assign inputs to classes in a classiication problem. Hidden Markov model A graphical probabilistic model characterized by a state vector, an output vector, a state transition matrix, an emission matrix and an initial state distribution. Likelihood function The probability of observing a particular data set under the assumption of a given parametrized model, expressed as a function of the adaptive parameters of the model. Mixture model A probability model which consists of a linear combination of simpler component probability models. Multilayer perceptron The most common form of neural network model, consisting of successive linear transformations followed by processing with nonlinear activation functions. Overrtting The problem in which a model which is too complex captures too much of the noise in the data, leading to poor generalization. Radial basis function network A common network model consisting of a linear combination of basis functions each of which is a function of the diierence between the input vector and a center vector. Regression A learning problem in which the goal is to map each input vector to a real-valued output vector. Regularization A technique for controlling model complexity and improving generalization by the addition of a penalty term to the cost function. In this chapter we have emphasized the links between neural networks and statistical pattern …