Neural Computation · 1993 · 761 citations · 15 references
ComputingComputer ScienceAlgorithmsMachine LearningStatistical LearningReinforcement LearningState-space ModelingState-space ModelsState-space RepresentationConstraintsNavigationSuccessor RepresentationTemporal-difference Learning
Estimation of returns over time, the focus of temporal difference (TD) algorithms, imposes particular constraints on good function approximators or representations. Appropriate generalization between states is determined by how similar their successors are, and representations should follow suit. This paper shows how TD machinery can be used to learn such representations, and illustrates, using a navigation task, the appropriately distributed nature of the result.
15
Christopher J. Watkins, Peter Dayan · Machine Learning · 1992 · 9K citations · Full text
Learning to Predict by the Methods of Temporal Differences
Richard S. Sutton · Machine Learning · 1988 · 4K citations · Full text