arXiv (Cornell University) · 2018 · 46 citations · 16 references
Natural SciencesMathematicsOptimizationProbability TheoryStochastic ProcessesConvex AnalysisConvex OptimizationGradient-based OptimizationStochastic CalculusStochastic OptimizationStochastic ProgrammingConvergence Rate AnalysisConvex FunctionsMoreau EnvelopesProximal AlgorithmsStochastic Gradient Descent
We prove that the proximal stochastic subgradient method, applied to a weakly convex problem, drives the gradient of the Moreau envelope to zero at the rate $O(k^{-1/4})$. As a consequence, we resolve an open question on the convergence rate of the proximal stochastic gradient method for minimizing the sum of a smooth nonconvex function and a convex proximable function.
16
Robust Stochastic Approximation Approach to Stochastic Programming
Arkadi Nemirovski, Anatoli Juditsky, Guanghui Lan et al. · SIAM Journal on Optimization · 2009 · 2.1K citations
Gradient methods for minimizing composite functions
Yu. Nesterov · Mathematical Programming · 2012 · 1.4K citations