A Selection Criterion for Robust Classifiers : Cancer Prognosis with Microarray Gene Expression
Microarray technology is now enabling us to measure the expression levels of large number of genes simultaneously. In clinical medicine, this technology is going to be applied to predict clinical features, such as malignancy of cancer [2]. For such purposes, we need classifiers which show a high prediction accuracy. However, estimation of the true prediction performance of a classifier is highly variable because of the small size of samples and non-negligible observation noise. Consequently, even when we have a classifier with high expected accuracy, it is not safe if its variance is high. We present here a new method, the Parametric Noise Bootstrap and Percentile (PNBP) method, to estimate the variance by generating artificial datasets which simulate an original dataset with observation noise. The advantages of PNBP are, firstly, we overcome the smallness of sample size to estimate the variance. Secondly, we set a clear criterion to select a robust and reliable classifier which does not depend on only one dataset but considers variance of classifier performance estimation with enough simulated datasets.