Continuum Regression: Cross-Validated Sequentially Constructed Prediction Embracing Ordinary Least Squares, Partial Least Squares and Principal Components Regression

TLDR

The study tackles the long‑standing challenge of selecting regressors for least‑squares multiple regression. The method introduces two tunable parameters—α (continuum [0,1]) and ω (number of regressors)—selected via cross‑validation and demonstrated across multiple examples. The approach reveals that ordinary least squares and principal components regression represent the extremes of a continuous spectrum, with partial least squares positioned between them.

Abstract

SUMMARY The paper addresses the evergreen problem of construction of regressors for use in least squares multiple regression. In the context of a general sequential procedure for doing this, it is shown that, with a particular objective criterion for the construction, the procedures of ordinary least squares and principal components regression occupy the opposite ends of a continuous spectrum, with partial least squares lying in between. There are two adjustable ‘parameters’ controlling the procedure: ‘alpha’, in the continuum [0, 1], and ‘omega’, the number of regressors finally accepted. These control parameters are chosen by cross-validation. The method is illustrated by a range of examples of its application.