A Normative Model of Attention: Modulation of Neural Response
When a sensory stimulus is encoded in a lossy fashion for ef- ficient transmission, there are necessarily tradeoffs between the represented fidelity of various aspects of the input pat- tern. In the model of attention presented here, a top-down signal informs the encoder of these tradeoffs. Given an en- semble of input patterns and tradeoff requirements, our sys- tem can learn to encode its inputs optimally. This general model is instantiated in a simple network: an autoencoder with a bottleneck, innervated by a top-down attentional sig- nal, trained using backpropagation. The only information the encoder receives concerning the semantics of the top-down attentional signal is from the optimization criterion, which penalizes the system more heavily for errors made near a simple attentional spotlight. The modulation of neural activ- ity learned by this model qualitatively matches that measured in animals during covert visual attention tasks.