2012 · 40 citations · 14 references
BusinessDecision SciencesAlgorithmsMarkov Decision ProcessesOptimizationConstrained Markov Decision ProcessesDynamic ProgrammingLinear ProgrammingPartially Observable Markov Decision ProcessesValue FunctionsValue IterationApproximate Dynamic ProgrammingConstrained Partially Observable Markov Decision ProcessesOptimal PoliciesPoint-based Value IterationPomdp SolversState-value Function
Constrained partially observable Markov decision processes (CPOMDPs) extend the standard POMDPs by allowing the specification of constraints on some aspects of the policy in addition to the optimality objective for the value function. CPOMDPs have many practical advantages over standard POMDPs since they naturally model problems involving limited resource or multiple objectives. In this paper, we show that the optimal policies in CPOMDPs can be randomized, and present exact and approximate dynamic programming methods for computing randomized optimal policies. While the exact method requires solving a minimax quadratically constrained program (QCP) in each dynamic programming update, the approximate method utilizes the point-based value update with a linear program (LP). We show that the randomized policies are significantly better than the deterministic ones. We also demonstrate that the approximate point-based method is scalable to solve large problems. 1
14