Learning Curved Multinomial Subfamilies for Natural Language Processing and Information Retrieval
Many problems in natural language learning and information retrieval involve estimating probabilities in very large discrete state spaces. Dimension reduction as well as clustering techniques in various avors have been popular choices to deal with the problem of data sparseness. In this paper, we present a general framework for dimension reduction based on curved multinomial subfamilies. The investigated class of models include di erent geometries as well as various objective functions and algorithms for model tting. The pursued goal is twofold { to achieve a systematic understanding of the di erences and similarities between various models and to empirically investigate their generalization performance on a number of representative data sets.