The HKU Scholars Hub (University of Hong Kong) · 2015 · 53 citations · 19 references
ComputingComputer ScienceMathematicsMachine LearningOptimizationStatistical LearningConvex OptimizationDigital Image ProcessingGradient-based OptimizationOptimization For Machine LearningAdaptive OptimizationAlternating Direction Method Of MultipliersConvergenceGradient DescentOptimal Convergence RatesOptimization AlgorithmsPrimal-dual AlgorithmsStatistical Learning Theory
The alternating direction method of multipliers (ADMM) is an important tool for solving complex optimization problems, but it involves minimization sub-steps that are often difficult to solve efficiently. The Primal-Dual Hybrid Gradient (PDHG) method is a powerful alternative that often has simpler sub-steps than ADMM, thus producing lower complexity solvers. Despite the flexibility of this method, PDHG is often impractical because it requires the careful choice of multiple stepsize parameters. There is often no intuitive way to choose these parameters to maximize efficiency, or even achieve convergence. We propose self-adaptive stepsize rules that automatically tune PDHG parameters for optimal convergence. We rigorously analyze our methods, and identify convergence rates. Numerical experiments show that adaptive PDHG has strong advantages over non-adaptive methods in terms of both efficiency and simplicity for the user.
19
The Split Bregman Method for L1-Regularized Problems
Tom Goldstein, Stanley Osher · SIAM Journal on Imaging Sciences · 2009 · 4.3K citations