Support vector regression performance analysis and systematic parameter selection

Abstract

Support vector regression (SVR) based on statistical learning is a useful tool for nonlinear regression problems. The SVR method deals with data in a high dimension space by using linear quadratic programming techniques. As a consequence, the regression result has optimal properties. However, if parameters were not properly selected, overfitting and/or underfilling phenomena might occur in SVR. Two parameters /spl sigma/, the width of Gaussian kernels and /spl epsi/, the tolerance zone in the cost function are considered in this research. We adopted the concept of the sampling theory into Gaussian filter to deal with parameter /spl sigma/. The idea is to analyze the frequency spectrum of training data and to select a cut-off frequency by including 90% of power in spectrum. The corresponding /spl sigma/ can then be obtained through the sampling theory. In our simulations, it can be found that good performances are observed when the selected frequency is near the cut-off frequency. For another parameter /spl epsi/, it is a tradeoff between the number of support vectors and the RMSE. By introducing the confidence interval concept, a suitable selection of /spl epsi/ can be obtained. The idea is to use the L/sub 1/-norm (i.e., when /spl epsi/ = 0 ) to estimate the noise distribution of training data. When /spl epsi/ is obtained by selecting the 90% confidence interval, simulations demonstrated superior performance in our illustrative example. By our systematical design, proper values of /spl sigma/ and /spl epsi/ can be obtained and the resultant system performances are nice in all aspects.