Sparse Spectral-based Feature Selection with Side Information
We address the problem of selecting a subset of the most relevant features from a set of sample data in cases where there are multiple (equally reasonable) solutions. In particular, this topic includes the suppression of one of the solutions given “side” data, i.e., when one is given information about undesired aspects of the data. Such situations often arise when there are several, even conflicting, dimensions to the data. For example, documents can be clustered based on topic, authorship or writing style; images of human faces can be clustered based on illumination conditions, facial expressions or by person identity; gene expressions levels can be clustered by pathologies or by correlations that also exist in other conditions, and so forth. We address this problem by extending the recently introduced algorithm for feature selection [10]. The algorithm uses spectral properties of the affinity (or Laplacian) matrix of the input data sample to guide the search for a relevant subset of features (coordinates). The most relevant subset of coordinates is the one that induces the most coherent collection of clusters in the subspace defined by the selected coordinates. In this work we address the problem which arises when different coordinate selections induce equally good clusters in their respective subspaces. We derive a principled manner to represent unwanted cluster arrangements, such as “side” data in various forms, as part of the optimization functional for feature selection and also derive a topographical structure of all selected feature subsets.