Neural Information Processing Systems · 1997 · 58 citations · 11 references
ComputingComputer ScienceMathematicsAlgorithmsMachine LearningOptimizationBias-variance TradeoffConvex OptimizationOptimization For Machine LearningOptimization TheoryRegressionLeast Squares EstimationLinear PredictionLinear RegressionRidge Regression
We apply a general algorithm for merging prediction strategies (the Aggregating Algorithm) to the problem of linear regression with the square loss; our main assumption is that the response variable is bounded. It turns out that for this particular problem the Aggregating Algorithm resembles, but is slightly different from, the well-known ridge estimation procedure. From general results about the Aggregating Algorithm we deduce a guaranteed bound on the difference between our algorithm's performance and the best, in some sense, linear regression function's performance. We show that the AA attains the optimal constant in our bound, whereas the constant attained by the ridge regression procedure in general can be 4 times worse.
11
The Weighted Majority Algorithm
N. Littlestone, Manfred K. Warmuth · Information and Computation · 1994 · 2K citations
Volodimir G. Vovk · Conference on Learning Theory · 1990 · 588 citations