LP and QP based learning from empirical data

Abstract

The quadratic programming (QP) and the linear programming (LP) based method are recently the most popular methods for learning from empirical data (observations, samples, examples, records). Support vector machines (SVMs) are the newest models based on the QP algorithm in solving nonlinear regression and classification problems. The LP based learning also controls both the number of basis functions in a neural network (i.e., support vector machine) and the accuracy of learning machine. Both methods result in a parsimonious network. This results in data compression. Two different methods are compared in terms of number of SVs (possible compression achieved) and in generalization capability. We compare the LP and QP based approaches by using the regression examples.