Regression Models for Ordinal Data: A Machine Learning Approach
In contrast to the standard machine learning tasks of classi fication and metric regression we investigate the problem of predicting var iables of ordinal scale, a setting referred to as ordinal regression. The task of ordinal regression arises frequently in the social sciences and in information retrie val where human preferences play a major role. Also many multi–class problems ar e really problems of ordinal regression due to an ordering of the classes. Alth oug the problem is rather novel to the Machine Learning Community it has been wi dely considered in Statistics before. All the statistical methods rely on a p robability model of a latent (unobserved) variable and on the condition of st chastic ordering . In this paper we develop a distribution independent formulation of the problem and give uniform bounds for our risk functional. The main difference to classification is the restriction that the mapping of objects to ranks must be tran sitive and asymmetric. Combining our theoretical framework with results from measurement theory we present an approach that is based on a mapping from objects to scalar utility values and thus guarantees transitivity and asymmetry. App lying the principle of Structural Risk Minimization as employed in Support Vector Machines we derive a new learning algorithm based on large margin rank boundari es for the task of ordinal regression. Our method is easily extended to nonlinea r utility functions. We give experimental results for an Information Retrieval tas k of learning the order of documents with respect to an initial query. Moreover, we sho w that our algorithm outperforms more naive approaches to ordinal regression su ch as Support Vector Classification and Support Vector Regression in the case of m ore than two ranks . 1This paper is a preliminary version of (Herbrich et al. 1999)