arXiv (Cornell University) · 2021 · 10 citations · 41 references
ComputingAutonomous SystemsComputer ScienceMachine LearningMarkov Decision ProcessesRobot LearningDeep LearningMulti-agent Robot LearningMulti-task LearningPolicy LearningReinforcement LearningTemporal LogicDeep Reinforcement LearningFormal Language TheoryLinear Temporal LogicLinear-time Temporal LogicMulti-task Deep LearningMulti-task Reinforcement LearningSample-efficient Policy Learning
We address the problem of teaching a deep reinforcement learning (RL) agent to follow instructions in multi-task environments. Instructions are expressed in a well-known formal language -- linear temporal logic (LTL) -- and can specify a diversity of complex, temporally extended behaviours, including conditionals and alternative realizations. Our proposed learning approach exploits the compositional syntax and the semantics of LTL, enabling our RL agent to learn task-conditioned policies that generalize to new instructions, not observed during training. To reduce the overhead of learning LTL semantics, we introduce an environment-agnostic LTL pretraining scheme which improves sample-efficiency in downstream environments. Experiments on discrete and continuous domains target combinatorial task sets of up to $\sim10^{39}$ unique tasks and demonstrate the strength of our approach in learning to solve (unseen) tasks, given LTL instructions.
41
The Graph Neural Network Model
Franco Scarselli, M. Gori, Ah Chung Tsoi et al. · IEEE Transactions on Neural Networks · 2008 · 9.3K citations · Full text
The temporal logic of programs
Amir Pnueli · 1977 · 5.7K citations
Model Checking of Safety Properties
Orna Kupferman, Moshe Y. Vardi · Formal Methods in System Design · 2001 · 577 citations