Duality and Auxiliary Functions for Bregman Distances

Abstract

Bregman distances provide a general framework in which many machine learning and statistical inference methods can be cast, with examples ranging from least squares regression and decision trees to logistic regression and support vector machines. This paper formulates and proves a convex duality theorem for Bregman distances and presents a technique based on auxiliary functions for deriving and proving convergence of iterative algorithms to minimize Bregman distance subject to linear constraints.