A neural network model for facial affect classification
This thesis presents an automated emotion recognition model that is capable of identifying six basic emotions (happy, surprise, sad, angry, fear, disgust) achieving an 86% generalization rate on novel face images. Faces are represented to the model by projecting local feature regions around the eyes and mouth on to principal components of random image blocks. The model is an ensemble of neural network classifiers that learn to recognize emotion by incrementally adjusting their responses to positive examples of the emotions. 
The model exhibits substantial agreement with humans in the confusions shown by the model between responses to portrayed emotions when averaged over many examples. When model and human face judgments are examined using multi-dimensional scaling, they result in a circular distribution of responses clustered by emotion in the same relative sequence. Comparison of the rankings derived from ordering the entries in the confusion matrix of humans and the model by emotion also shows significant agreement. This is true even when similarity information is masked from the model during training. 
The model reproduces behavior seen in human subjects when they are presented with interpolated face images between two emotion prototypes of a single face. Categorical perception is observed in humans in this situation. The neural network model generates similar behavior when evaluating these images. It also exhibits the same type of category structure in that its judgments about the mix of emotions present in the interpolated images mirrors that of humans. 
Three conclusions are drawn from these results. First, discrete training with expressive emotion prototypes (as compared to training with continuous stimuli) is sufficient to learn an emotion space that captures many human characteristics. Second, local unbiased features, concentrated in regions rich in variations (eyes and mouth) provide for better generalization than do either global features or local features specifically tuned to those regions. Finally, we conclude that categorical perception (CP) in human emotion responses to continuous face stimuli, can be learned by a simple classifier. This suggests that CP is not an intrinsic property of biological processing but is the result of learning in a distributed, non-linear decision system.