Parcel: Feature Subset Selection in Variable Cost Domains
The vast majority of classification systems are designed with a single set of features, and optimised to a single specified cost. However, in examples such as medical and financial risk modelling, costs are known to vary subsequent to system design. In this paper, we present a design method for feature selection in the presence of varying costs. Starting from the Wilcoxon nonparametric statistic for the performance of a classification system, we introduce a concept called the maximum realisable receiver operating characteristic ( ), and prove a related theorem. A novel criterion for feature selection, based on the area under the curve, is then introduced. This leads to a framework which we call Parcel. This has the flexibility to use different combinations of features at different operating points on the resulting curve. Empirical support for each stage in our approach is provided by experiments on real world problems, with Parcel achieving superior results.