Journal of the American Statistical Association · 2024 · 12 citations · 30 references
Natural SciencesData ScienceCausal InferenceEconometricsMachine LearningBias-variance TradeoffEstimation TheoryOutlier HandlingRobust Statistical ModelingRobust StatisticsStatistical InferenceContamination ModelsHuber EstimationLeast Trimmed SquaresLinear RegressionPostprocessing TechniquesRobust EstimationRobust RegressionStatistical Postprocessing
We study the problem of linear regression where both covariates and responses are potentially (i) heavy-tailed and (ii) adversarially contaminated. Several computationally efficient estimators have been proposed for the simpler setting where the covariates are sub-Gaussian and uncontaminated; however, these estimators may fail when the covariates are either heavy-tailed or contain outliers. In this work, we show how to modify the Huber regression, least trimmed squares, and least absolute deviation estimators to obtain estimators which are simultaneously computationally and statistically efficient in the stronger contamination model. Our approach is quite simple, and consists of applying a filtering algorithm to the covariates, and then applying the classical robust regression estimators to the remaining data. We show that the Huber regression estimator achieves near-optimal error rates in this setting, whereas the least trimmed squares and least absolute deviation estimators can be made to achieve near-optimal error after applying a postprocessing step. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.
30
Robust Estimation of a Location Parameter
Peter J. Huber · The Annals of Mathematical Statistics · 1964 · 7K citations · Full text
Robust Statistics: The Approach Based on Influence Functions
David Ruppert, Frank R. Hampel, Elvezio Ronchetti et al. · Technometrics · 1987 · 3.8K citations
Least Median of Squares Regression
Peter J. Rousseeuw · Journal of the American Statistical Association · 1984 · 3.6K citations
Robust Regression: Asymptotics, Conjectures and Monte Carlo
Peter J. Huber · The Annals of Statistics · 1973 · 2.4K citations · Full text