Mixtures of Independent Component Analysers
Gaussian Mixture Models (GMMs) are widely used throughout the fields of machine learning and statistics. Despite their popularity, however, GMMs suffer from two serious drawbacks. The first is that, as the dimensionality d of the problem space increases, the size of each covariance matrix, d 2, becomes prohibitively large. This can be dealt with by assuming isotropic Gaussians (ie.ignoring the covariance structure) but this greatly reduces the flexibility of the model class. This problem has been solved by Tipping and Bishop [12] who replaced each Gaussian with a probabilistic Principal Component Analysis (PCA) model. This allowed the dimensionality of each covariance to be effectively reduced whilst maintaining the richness of the model class. The second problem with GMMs is that each component is a Gaussian, an assumption which is often violated in many natural clustering problems [5]. It is this second problem which we address in this paper. A solution is reached by extending the mixtures of probabilistic PCA model to a mixtures of Independent Component Analysis (ICA) model.