An Introduction to Stochastic Neural Networks an Introduction to Stochastic Neural Net- Works
x1 Introduction How does the brain compute? Particularly in the last hundred years have we gathered an enormous amount of experimental ndings that shed some light on this question. The picture that has emerged is that the neuron is the central computing element of the brain which performs a non-linear input to output mapping between its synaptic inputs and its spiky output. The neurons are connected by synaptic junctions, thus forming a neural network. A central question is how such a neural network implements brain functions such as vision, audition and motor control. These questions are to a certain extend premature, because our knowledge of the functioning of the neuron and the synaptic process itself is only partial and much remains to be discovered. Nevertheless, it is interesting to see what emergent behavior arises in a network of very simple neurons. The pioneering work in this direction was done by McCulloch and Pitts 1) in the '40s. Taking the thresholding property of neurons to the extreme, they proposed that neurons perform logical operations on their inputs, such as AND and OR. One can show that a network of such neurons, when properly wired, can perform any logical function and is equivalent to a Turing machine. When considering neural networks, an important distinction is between feed-forward networks and recurrent networks. In feed-forward networks, the neurons can be labeled such that each neuron only receives input from neurons with lower label. Thus, one can identify input neurons, which receive no input from other neurons and whose activity depends only on the sensory stimulus,