Support vector regression with applications in automated drug discovery
The purpose of this research is to develop effective machine learning or data mining techniques based on support vector machines (SVMs), and apply them to practical problems arising from computer-aided drug discovery. SVMs are a robust inference methodology originally derived for pattern recognition and later extended to other inference problems such as regression problems. In this study, we investigate novel approaches for support vector regression (SVR), variable selection and model evaluation. First, an intuitive geometric framework for SVR is developed based on duality analysis from the geometric and mathematical programming perspective. We show that SVR can be regarded as a classification problem in the dual space. The regression function is constructed by separating the convex or reduced convex hulls respectively of the training data with the response variable shifted up and down by an error tolerance e. A geometric SVR formulation is proposed based on choosing the maximal margin plane between the two shifted datasets. Second, we develop a variable selection method to address challenging drug design problems where problems usually have a relatively small number of molecules (observations) but a large number of descriptors (variables). The basic idea is to construct a series of linear models that exhibit good generalization by solving sparse linear SVMs. The subset of variables having nonzero weights in the linear models is used in nonlinear SVM to produce the final regression model. The method exploits the fact that a linear SVM (no kernels) with e 1-norm regularization inherently performs variable selection as a side-effect of minimizing capacity in the SVM method. Third, we devise a new technique for evaluation and comparison of regression models. The proposed approach, regression error characteristics (REC) curves, depicts the trade-off of the error tolerance versus the accuracy of the functions. REC curves generalize the ROC curve evaluation method for classification to regression. The REC curve estimates the error cumulative distribution function. Commonly-used error measures can be visually represented using the geometry of the REC plot.