Concepedia

Multichain Markov Decision Processes

Multichain Markov Decision Processes is a class of Markov decision processes characterized by the presence of multiple recurrent classes under deterministic stationary policies, where the transition structure allows for several closed communicating sets of states and transient states, distinguishing it from unichain processes by enabling non-uniform ergodic behavior across policies.

22

Publications

987

Citations

29

Authors

26

Institutions

Publications per year

2017–2026

1

Variants

Multichain Mdps

Parents

Publications

22

Multichain Markov Renewal Programs

Eric V. Denardo, Bennett L. Fox · SIAM Journal on Applied Mathematics · 1968 · 156 citations

, , +16

Constrained Undiscounted Stochastic Dynamic Programming

Arie Hordijk, L. C. M. Kallenberg · Mathematics of Operations Research · 1984 · 109 citations

, , +14

Multichain Markov Decision Processes with a Sample Path Constraint: A Decomposition Approach

Keith W. Ross, Ravi Varadarajan · Mathematics of Operations Research · 1991 · 91 citations

, , +16

Markov Renewal Programming by Linear Fractional Programming

Bennett L. Fox · SIAM Journal on Applied Mathematics · 1966 · 89 citations

, , +13

Multiple-goal reinforcement learning with modular Sarsa(O)

Nathan Sprague, Dana H. Ballard · 2003 · 84 citations

, , +11

1–5 of 22

Authors

29

  1. AF
    Awi Federgruen

    Columbia University, United States

    3 publications · 103 citations

    3

    h-index

  2. PJ
    Paul J. Schweitzer

    University of Rochester, United States

    3 publications · 103 citations

    3

    h-index

  3. AH
    Arie Hordijk

    Leiden University, Netherlands

    3 publications · 161 citations

    3

    h-index

  4. KW
    Keith W. Ross

    New York University, United States

    2 publications · 120 citations

    2

    h-index

  5. MB
    Melike Baykal‐Gürsoy

    Rutgers, The State University of New Jersey, United States

    2 publications · 53 citations

    2

    h-index

1–5 of 29

Institutions

26

  1. IBM Research - Thomas J. Watson Research Center

    Yorktown Heights, United States

    3 h-index

    3 publications

    103 citations

  2. University of Pennsylvania

    Philadelphia, United States

    2 h-index

    2 publications

    120 citations

  3. Rutgers, The State University of New Jersey

    New Brunswick, United States

    2 h-index

    2 publications

    53 citations

  4. University of Rochester

    Rochester, United States

    2 h-index

    3 publications

    216 citations

  5. Hong Kong University of Science and Technology

    Hong Kong, Hong Kong

    2 h-index

    3 publications

    81 citations

  6. Tsinghua University

    Beijing, China

    2 h-index

    4 publications

    94 citations

  7. Stony Brook University

    Stony Brook, United States

    1 h-index

    1 publications

    68 citations

  8. Chinese Academy of Sciences

    Beijing, China

    1 h-index

    1 publications

    28 citations

1–8 of 26

Venues

  1. Mathematics of Operations Research

    7 publications · 351 citations

    7

    h-index

  2. SIAM Journal on Applied Mathematics

    2 publications · 245 citations

    2

    h-index

  3. IEEE Transactions on Automatic Control

    2 publications · 52 citations

    2

    h-index

  4. Automatica

    1 publications · 19 citations

    1

    h-index

  5. Journal of Futures Markets

    1 publications · 18 citations

    1

    h-index

1–5