A scalable modular convex solver for regularized risk minimization

TLDR

Regularized risk minimization underlies many machine learning tasks such as SVMs, logistic regression, CRFs, and Lasso. This work presents a highly scalable, modular convex solver that addresses all these estimation problems. The solver is parallelizable across workstation clusters, supports data‑locality, and handles both l1 and l2 regularizers. It implements 20 estimation problems, scales to millions of observations, is up to ten times faster than specialized solvers, and its open‑source code is available in the ELEFANT toolbox.

Abstract

A wide variety of machine learning problems can be described as minimizing a regularized risk functional, with different algorithms using different notions of risk and different regularizers. Examples include linear Support Vector Machines (SVMs), Logistic Regression, Conditional Random Fields (CRFs), and Lasso amongst others. This paper describes the theory and implementation of a highly scalable and modular convex solver which solves all these estimation problems. It can be parallelized on a cluster of workstations, allows for data-locality, and can deal with regularizers such as l1 and l2 penalties. At present, our solver implements 20 different estimation problems, can be easily extended, scales to millions of observations, and is up to 10 times faster than specialized solvers for many applications. The open source code is freely available as part of the ELEFANT toolbox.