USING UPPER BOUNDS ON ATTAINABLEDISCRIMINATION TO SELECT DISCRETEVALUED
Selection of features that will permit accurate pattern clas-siication is, in general, a diicult task. However, if a particular data set is represented by discrete valued features, it becomes possible to determine empirically the contribution that each feature makes to the discrimination between classes. We describe how to calculate the maximum discrimination possible in a two alternative forced choice decision problem, when discrete valued features are used to represent a given data set. (In this paper, we measure discrimination in terms of the area under the receiver operating characteristic (ROC) curve.) Since this bound corresponds to the upper limit of classiication achievable by any classiier (with that given data representation), we can use it to assess whether recognition errors are due to a lack of separability in the data or shortcomings in the classiication technique. In comparison to the training and testing of artiicial neural networks, the empirical bound on discrimination can be eeciently calculated , allowing an experimenter to decide whether subsequent development of neural network models is warranted. We extend the discrimination bound method so that we can estimate both the maximum and average discrimination we can expect on unseen test data. These estimation techniques are the basis of a backwards elimination algorithm that can be used to rank features in order of their discriminative power. We use two problems to demonstrate this feature selection process: classiication of the Mushroom Database, and a real-world, pregnancy related medical risk prediction task | assessment of risk of perinatal death.