Support vector density estimation
We describe how the SV technique of solving linear operator equations can be applied to the problem of density estimation and how this method makes use of a special type of problem-speciic regularization. We present a new optimization procedure and set of kernels that guarantee the estimate to be a density (be non-negative everywhere and have an integral of 1). We introduce a dictionary of kernel functions to nd approximations using kernels of diierent widths adaptively. A method of SV regression using square loss is introduced and it is shown how this technique is useful for density estimation. Finally, a way of compressing density estimates from classical kernel based methods is described, and all these algorithms are compared to classical kernel density estimates (Parzen's windows). 1.1 The density estimation problem We wish to approximate the density function p(x) from data where the corresponding distribution function is F(x) = P(X x) = Z x ?1 p(t)dt: (If not speciied otherwise our densities are with respect to the usual Lebesgue measure.) Finding the required density means solving the linear operator equation 1 Z 1 ?1 (x ? t)p(t)dt = F(x); (1.1) 1.