Reduction of a Controlled Markov Model with Incomplete Data to a Problem with Complete Information in the Case of Borel State and Control Space

Abstract

Previous article Next article Reduction of a Controlled Markov Model with Incomplete Data to a Problem with Complete Information in the Case of Borel State and Control SpaceA. A. YushkevichA. A. Yushkevichhttps://doi.org/10.1137/1121014PDFBibTexSections ToolsAdd to favoritesExport CitationTrack CitationsEmail SectionsAbout[1] A. N. Shiryaev, On the theory of decision functions and control of an observation process with incomplete data, Trans. Third Prague Confer. on Inform. Theory etc., Prague, 1964, 657–682, (In Russian.) 0207.46705 Google Scholar[2] A. N. Širjaev, Some new results in the theory of controlled random processesTrans. Fourth Prague Conf. on Information Theory, Statistical Decision Functions, Random Processes (Prague, 1965), Academia, Prague, 1967, 131–203, (In Russian.) MR0225593 Google Scholar[3] E. B. Dynkin, Controlled random sequences, Theory Prob. Applications, 10 (1965), 1–14 10.1137/1110001 LinkGoogle Scholar[4] K. Hinderer, Foundations of non-stationary dynamic programming with discrete time parameter, Lecture Notes in Operations Research and Mathematical Systems, Vol. 33, Springer-Verlag, Berlin, 1970vi+160 MR0267890 0202.18401 CrossrefGoogle Scholar[5] Yoshikazu Sawaragi and , Tsuneo Yoshikawa, Discrete-time Markovian decision processes with incomplete state observation, Ann. Math. Statist., 41 (1970), 78–86 MR0261730 0188.48102 CrossrefGoogle Scholar[6] K. R. Parthasarathy, Probability measures on metric spaces, Probability and Mathematical Statistics, No. 3, Academic Press Inc., New York, 1967xi+276, London MR0226684 0153.19101 CrossrefGoogle Scholar[7] Ralph E. Strauch, Negative dynamic programming, Ann. Math. Statist., 37 (1966), 871–890 MR0194243 0144.43201 CrossrefGoogle Scholar[8] E. B. Dynkin, Foundations of the Theory of Markov Processes, Fizmatigiz, Moscow, 1959, (In Russian.) Translated into German: Die Grundlagen der Theorie der Markoffschen Prozesse, Springer, Berlin, 1961. Google Scholar[9A] Nicolas Bourbaki, Elements of mathematics. General topology. Part 1, Hermann, Paris, 1966vii+437 MR0205210 Google Scholar[9B] Nicolas Bourbaki, Elements of mathematics. General topology. Part 2, Hermann, Paris, 1966iv+363 MR0205211 Google Scholar[10] Detlef Rhenius, Incomplete information in Markovian decision models, Ann. Statist., 2 (1974), 1327–1334 MR0378840 CrossrefGoogle Scholar Previous article Next article FiguresRelatedReferencesCited byDetails Markov Decision Processes with Incomplete Information and Semiuniform Feller Transition ProbabilitiesEugene A. Feinberg, Pavlo O. Kasyanov, and Michael Z. Zgurovsky22 August 2022 | SIAM Journal on Control and Optimization, Vol. 60, No. 4AbstractPDF (568 KB)Partially Observed Discrete-Time Risk-Sensitive Mean Field Games7 June 2022 | Dynamic Games and Applications, Vol. 156 Cross Ref Robustness to Incorrect Priors and Controlled Filter Stability in Partially Observed Stochastic ControlCurtis McDonald and Serdar Yüksel4 April 2022 | SIAM Journal on Control and Optimization, Vol. 60, No. 2AbstractPDF (546 KB)Convergence and Near Optimality of Q-Learning with Finite Memory for Partially Observed Models Cross Ref A Class of Solvable Markov Decision Models with Incomplete Information Cross Ref History-dependent Evaluations in Partially Observable Markov Decision ProcessXavier Venel and Bruno Ziliotto29 April 2021 | SIAM Journal on Control and Optimization, Vol. 59, No. 2AbstractPDF (544 KB)Stochastic Control with Complete Observations on a Finite Horizon3 August 2021 Cross Ref Stochastic Control with Partial Observations on a Finite Horizon3 August 2021 Cross Ref A Universal Dynamic Program and Refined Existence Results for Decentralized Stochastic ControlSerdar Yüksel8 September 2020 | SIAM Journal on Control and Optimization, Vol. 58, No. 5AbstractPDF (519 KB)Asymptotic Optimality of Finite Model Approximations for Partially Observed Markov Decision Processes With Discounted CostIEEE Transactions on Automatic Control, Vol. 65, No. 1 Cross Ref Weak Feller property of non-linear filtersSystems & Control Letters, Vol. 134 Cross Ref Partially-Observed Discrete-Time Risk-Sensitive Mean-Field Games Cross Ref On Weak Feller Continuity Properties of Non-linear Filters Cross Ref Approximate Nash Equilibria in Partially Observed Stochastic Games with Mean-Field InteractionsMathematics of Operations Research, Vol. 44, No. 3 Cross Ref Robustness to Incorrect Priors in Partially Observed Stochastic ControlAli̇ Devran Kara and Serdar Yüksel28 May 2019 | SIAM Journal on Control and Optimization, Vol. 57, No. 3AbstractPDF (593 KB)Approximations for Partially Observed Markov Decision Processes12 May 2018 Cross Ref Partially Observable Risk-Sensitive Markov Decision ProcessesMathematics of Operations Research, Vol. 42, No. 4 Cross Ref Stochastic switching for partially observable dynamics and optimal asset allocation15 June 2016 | International Journal of Control, Vol. 90, No. 3 Cross Ref Uniform Fatou's lemmaJournal of Mathematical Analysis and Applications, Vol. 444, No. 1 Cross Ref Optimality Conditions for Inventory Control4 November 2016 Cross Ref Partially Observable Total-Cost Markov Decision Processes with Weakly Continuous Transition ProbabilitiesMathematics of Operations Research, Vol. 41, No. 2 Cross Ref Near optimality of quantized policies in stochastic control under weak continuity conditionsJournal of Mathematical Analysis and Applications, Vol. 435, No. 1 Cross Ref Strong Uniform Value in Gambling Houses and Partially Observable Markov Decision ProcessesXavier Venel and Bruno Ziliotto9 August 2016 | SIAM Journal on Control and Optimization, Vol. 54, No. 4AbstractPDF (501 KB)Convergence of value iterations for total-cost MDPs and POMDPs with general state and action sets Cross Ref Asymptotic optimality of quantized policies in stochastic control under weak continuity conditions Cross Ref Optimality Conditions for Partially Observable Markov Decision Processes26 November 2013 Cross Ref Optimality conditions for total-cost Partially Observable Markov Decision Processes Cross Ref Blackwell optimality in Markov decision processes with partial observationThe Annals of Statistics, Vol. 30, No. 4 Cross Ref Discrete-Time Controlled Markov Processes with Average Cost Criterion: A SurveyAristotle Arapostathis, Vivek S. Borkar, Emmanuel Fernández-Gaucherand, Mrinal K. Ghosh, and Steven I. Marcus14 July 2006 | SIAM Journal on Control and Optimization, Vol. 31, No. 2AbstractPDF (7400 KB)Nonparametric adaptive control of discrete-time partially observable stochastic systemsJournal of Mathematical Analysis and Applications, Vol. 137, No. 2 Cross Ref Adaptive control of Markov processes with incomplete state information and unknown parametersJournal of Optimization Theory and Applications, Vol. 52, No. 2 Cross Ref Optimal resource extraction and uncertainty about new technologyJournal of Environmental Economics and Management, Vol. 11, No. 1 Cross Ref The Importance of Learning in Planning with Exhaustible Resources Cross Ref REMOVED: References Cross Ref Volume 21, Issue 1| 1976Theory of Probability & Its Applications History Submitted:10 October 1974Published online:17 July 2006 InformationCopyright © 1976 Society for Industrial and Applied MathematicsPDF Download Article & Publication DataArticle DOI:10.1137/1121014Article page range:pp. 153-158ISSN (print):0040-585XISSN (online):1095-7219Publisher:Society for Industrial and Applied Mathematics