Mathematics of Operations Research · 2007 · 50 citations · 18 references
Natural SciencesComputer ScienceMathematicsAlgorithmsOptimizationSchedulingOptimal ControlOptimization Under UncertaintyStochastic CalculusStochastic ControlStochastic OptimizationStochastic ProgrammingApproximate Dynamic ProgrammingConvergenceDescentDiscretizationElectrical Power SystemsOptimal Control Of Stochastic SystemsStochastic Dynamic ProgrammingStochastic Gradient Descent
We propose a Hilbert-valued perturbed subgradient algorithm with stochastic noises, and we provide a convergence proof for this algorithm under classical assumptions on the descent direction and new assumptions on the stochastic noises. Instead of requiring the stochastic noises to correspond to martingale increments, we only require these noises to be asymptotically so. Furthermore, the variance of these noises is allowed to grow infinitely under the control of a decreasing sequence linked with the subgradient stepsizes. This algorithm can be used to solve stochastic closed-loop control problems without any a priori discretization of the uncertainty, such as linear decision rules or tree representations. It can also be used as a way to perform stochastic dynamic programming without state-space discretization or a priori functional bases (i.e., approximate dynamic programming). Both problems arise frequently—for example, in power systems scheduling or option pricing. This article focuses on the theorical foundations of the algorithm, and gives references to detailed practical experimentations.
18
A Stochastic Approximation Method
Herbert Robbins, Sutton Monro · The Annals of Mathematical Statistics · 1951 · 9.6K citations · Full text
Stochastic approximation: invited paper
Tze Leung Lai · The Annals of Statistics · 2003 · 128 citations · Full text