Deep Neural Networks Algorithms for Stochastic Control Problems on Finite Horizon: Convergence Analysis

Abstract

This paper develops algorithms for high-dimensional stochastic control\nproblems based on deep learning and dynamic programming. Unlike classical\napproximate dynamic programming approaches, we first approximate the optimal\npolicy by means of neural networks in the spirit of deep reinforcement\nlearning, and then the value function by Monte Carlo regression. This is\nachieved in the dynamic programming recursion by performance or hybrid\niteration, and regress now methods from numerical probabilities. We provide a\ntheoretical justification of these algorithms. Consistency and rate of\nconvergence for the control and value function estimates are analyzed and\nexpressed in terms of the universal approximation error of the neural networks,\nand of the statistical error when estimating network function, leaving aside\nthe optimization error. Numerical results on various applications are presented\nin a companion paper (arxiv.org/abs/1812.05916) and illustrate the performance\nof the proposed algorithms.\n