On the Adjoint Process for Optimal Control of Diffusion Processes

Abstract

For the Markovian control problem \[\begin{gathered} \mathop {\min }\limits_u E\left\{ {\int_0^T {l(t,x,u)dt + c(x(T))} } \right\}, \hfill \\ dx = f(t,x,u)dt + \sigma (t,x)dw, \hfill \\ \end{gathered} \] it is shown that the adjoint process appearing in the maximum principle has the form \[p(t,x) = - E_{tx} \left\{ {\frac{{\partial c}}{{\partial x}}(x(T))\Phi (T, t) + \int_t^T {\frac{{\partial l}}{{\partial x}}(s,x(s),\hat u(s,x(s))\Phi (s,t)ds} } \right\},\] where $\hat u$ is the optimal feedback control, $E_{tx} $ denotes conditional expectation, and $\Phi $ is the fundamental matrix solution of \[dy = \frac{{\partial f}}{{\partial x}}(t,x(t)\hat u(t,x(t))),ydt + \sum_k {\frac{{\partial \sigma ^k }}{{\partial x}}(t,x))y} dw_k \] Here, $\sigma ^k $ is the kth column of $\sigma $ and $w_k $ is thekth component of the Brownian motion w, It is also shown that $p(t,x(t))$ satisfies ($ * $ denotes transpose) \[\begin{gathered} dp^ * = \left\{ - {\frac{{\partial f^ * }}{{\partial x}}(t,x(t),\hat u(t,x(t)))p^ * + \sum\limits_k {\frac{{\partial \sigma ^k }}{{\partial x}}(t,x(t))V_{xx} (t,x(t))\sigma ^k (t,x(t))} + \frac{{\partial l^ * }}{{\partial x}}(t,x(t),\hat u(t,x(t)))} \right\}dt - V_{xx} (t,x(t))\sigma (t,x(t))dw, \hfill \\ p(T) = - \frac{{\partial c}}{{\partial x}}(x,(T)), \hfill \\ \end{gathered} \] where $V_{xx} $ is the Hessian of the value function V.