Ju n 20 05 Efficient Multiclass Implementations of L 1-Regularized Maximum Entropy
This paper discusses the application of L1-regularized max i um entropy modeling or SL1-Max[9] to multiclass categorization problems. A new modification t the SL1-Max fast sequential learning algorithm is proposed to handle conditionaldistributions. Furthermore, unlike most previous studies , the present research goes beyond a single type of conditiona l distribution. It describes and compares a variety of modeling assumptions about the class distributi on (independent or exclusive) and various types of joint or conditional distributions. It results in a new me thodology for combining binary regularized classifiers to achieve multiclass categorization. In this c ontext, Maximum Entropy can be considered as a generic and efficient regularized classification tool th at matches or outperforms the state-of-the art represented by AdaBoost and SVMs.