2003 · 26 citations · 6 references
EngineeringMathematicsDynamical SystemsMarkov Decision ProcessesStochastic ProcessesMarkov ProcessesPartially Observable Markov Decision ProcessesPredictive ModelingState-space ModelsState-space RepresentationCompressionNonlinear Dynamical Systems
Predictive state representations (PSRs) use predictions of a set of tests to represent the state of controlled dynamical systems. One reason why this representation is exciting as an alternative to partially observable Markov decision processes (POMDPs) is that PSR models of dynamical systems may be much more compact than POMDP models. Empirical work on PSRs to date has focused on linear PSRs, which have not allowed for compression relative to POMDPs. We introduce a new notion of tests which allows us to define a new type of PSR that is nonlinear in general and allows for exponential compression in some deterministic dynamical systems. These new tests, called e-tests, are related to the tests used by Rivest and Schapire [1] in their work with the diversity representation, but our PSR avoids some of the pitfalls of their representation—in particular, its potential to be exponentially larger than the equivalent POMDP. 1
6
Predictive Representations of State
Michael L. Littman, Richard S. Sutton · 2001 · 428 citations
Algorithms for Sequential Decision Making
Michael L. Littman · 1996 · 345 citations
Observable Operator Models for Discrete Stochastic Time Series
Herbert Jaeger · Neural Computation · 2000 · 181 citations
Learning predictive state representations
Satinder Singh, Michael L. Littman, Nicholas K. Jong et al. · 2003 · 113 citations