Model Induction with Support Vector Machines: Introduction and Applications

TLDR

Recent advances in information processing have driven engineering research toward intelligent systems that automatically model natural phenomena, with many machine‑learning techniques already applied to civil engineering systems. This study investigates the use of support vector machines, a statistical‑learning paradigm, for automatic model induction. The authors review statistical‑learning theory and SVM, illustrate its application to two empirical model‑induction cases, and compare its performance with artificial neural networks.

Abstract

The rapid advance in information processing systems in recent decades had directed engineering research towards the development of intelligent systems that can evolve models of natural phenomena automatically—"by themselves," so to speak. In this respect, a wide range of machine learning techniques like decision trees, artificial neural networks (ANNs), Bayesian methods, fuzzy-rule based systems, and evolutionary algorithms have been successfully applied to model different civil engineering systems. In this study, the possibility of using yet another machine learning paradigm that is firmly based on the theory of statistical learning, namely that of the support vector machine (SVM), is investigated. An interesting property of this approach is that it is an approximate implementation of a structural risk minimization (SRM) induction principle that aims at minimizing a bound on the generalization error of a model, rather than minimizing only the mean square error over the data set. In this paper, the basic ideas underlying statistical learning theory and SVM are reviewed, and the potential of the SVM for feature classification and multiple regression (modeling) problems is demonstrated by applying the method to two different cases of model induction from empirical data. The relative performance of the SVM is then analyzed by comparing its results with that of ANNs on the same data sets.