Minimal Learning Machine: Theoretical Results and Clustering-Based\n Reference Point Selection

Abstract

The Minimal Learning Machine (MLM) is a nonlinear supervised approach based\non learning a linear mapping between distance matrices computed in the input\nand output data spaces, where distances are calculated using a subset of points\ncalled reference points. Its simple formulation has attracted several recent\nworks on extensions and applications. In this paper, we aim to address some\nopen questions related to the MLM. First, we detail theoretical aspects that\nassure the interpolation and universal approximation capabilities of the MLM,\nwhich were previously only empirically verified. Second, we identify the task\nof selecting reference points as having major importance for the MLM's\ngeneralization capability. Several clustering-based methods for reference point\nselection in regression scenarios are then proposed and analyzed. Based on an\nextensive empirical evaluation, we conclude that the evaluated methods are both\nscalable and useful. Specifically, for a small number of reference points, the\nclustering-based methods outperformed the standard random selection of the\noriginal MLM formulation.\n