ScholarWorks@UMassAmherst (University of Massachusetts Amherst) · 2007 · 231 citations · 23 references
The options framework enables RL agents to build high‑level skills, but options learned in a problem’s state space cannot be transferred to similar tasks with different state spaces. The study proposes learning options in agentspace—a feature space shared across tasks—rather than in problemspace. The authors learn options directly in agentspace, a feature space that remains consistent across tasks, enabling options to be reused across different problem spaces. Agent‑space options can be reused in later tasks that share the same agent‑space but have different problem‑spaces, and experimental results show they perform best when combined with problem‑space options.
The options framework provides methods for reinforcement learning agents to build new high-level skills. However, since options are usually learned in the same state space as the problem the agent is solving, they cannot be used in other tasks that are similar but have different state spaces. We introduce the notion of learning options in agentspace, the space generated by a feature set that is present and retains the same semantics across successive problem instances, rather than in problemspace. Agent-space options can be reused in later tasks that share the same agent-space but have different problem-spaces. We present experimental results demonstrating the use of agent-space options in building transferrable skills, and show that they perform best when used in conjunction with problem-space options.
23