Journal of Physics Conference Series · 2021 · 11 citations · 11 references
ComputingComputer ScienceData MiningMachine LearningStatistical LearningClassificationKernel MethodsLoss FunctionOptimization For Machine LearningSupervised LearningSupport Vector MachinesGradient DescentHinge LossRegularizationSmoothing TechniquesStochastic Gradient DescentSupport Vector ClassificationSupport Vector Machine ClassificationSvms
Abstract This work describes a framework for solving support vector machine with kernel (SVMK). Recently, it has been proved that the use of non-smooth loss function for supervised learning problem gives more efficient results [1]. This gives the idea of solving the SVMK problem based on hinge loss function. However, the hinge loss function is non-differentiable (we can’t use the standard optimization methods to minimize the empirical risk). To overcome this difficulty, a special smoothing technique for the hinge loss is proposed. Thus, the obtained smooth problem combined with Tikhonov regularization is solved using a stochastic gradient descent method. Finally, some numerical experiments on academic and real-life datasets are presented to show the efficiency of the proposed approach.
11
N. Aronszajn · Transactions of the American Mathematical Society · 1950 · 5.4K citations · Full text
Regularization Theory and Neural Networks Architectures
Federico Girosi, Michael Jones, Tomaso Poggio · Neural Computation · 1995 · 1.4K citations
Are Loss Functions All the Same?
Lorenzo Rosasco, Ernesto De Vito, Andrea Caponnetto et al. · Neural Computation · 2004 · 505 citations · Full text