Relaxed Clipping: A Global Training Method for Robust Regression and Classification

Abstract

Robust regression and classification are often thought to require non-convex functions that prevent scalable, global training. However, such a view neglects the possibility of reformulated training methods that can yield practically solvable alternatives. A natural way to make a function more robust to outliers is to truncate values that exceed a maximum threshold. We demonstrate that a relaxation of this form of loss clipping can be made globally solvable and applicable to any standard while guaranteeing robustness against outliers. We present a generic procedure that can be applied to standard functions and demonstrate improved robustness in regression and classification problems.