CONSTRAINED MAXIMUM LIKELIHOOD MODELING WITH GAUSSIAN DISTRIBUTIONS
Maximum Likelihood (ML) modeling of multiclass data using gaussian distributions for classification often suffers from the following problems: a) data insufficiency implying overtrained or unreliable models b) large storage requirement c) large computational requirement and/or d) ML is not discriminating between classes. Sharing parameters across classes (or constraining the parameters) clearly tends to alleviate the first three problems. It this paper we show that in some cases it can also lead to better discrimination (as evidenced by reduced misclassification error). The parameters considered are the means and variances of the gaussians and linear transformations of the feature space (or equivalently the gaussian means). Some forms of sharing (either explicit or implicit via constraints) on the parameters are shown to lead to Linear Discrimination Analysis (a well-known result) while others (like diagonal, block-diagonal and factor analyzed covariances) are shown to lead to optimal feature spaces. The key idea is that in constrained ML modeling one may be able to better model the data after it is linearly transformed, perhaps in a class dependent fashion. If the constrains are invariant to linear transformations (ILT), then, the original feature space is as good as any to model the data. Results using optimal feature spaces for diagonal covariances is shown using the speech recognition problem as an example.