2018 · 19 citations · 21 references
ComputingAutonomous SystemsComputer ScienceArtificial IntelligenceMachine LearningMarkov Decision ProcessesExploration-exploitation TradeoffOnline AlgorithmsOptimization For Machine LearningPolicy LearningReinforcement LearningReinforcement Learning FoundationsAction Space FormulationAgent Reinforcement LearningControl ActionsReinforcement Learning TheoryResearch Agenda SettingTemporal Reward AssignmentWorkload
In this paper we define a research agenda to develop a general framework supporting online autonomous tuning of data partitioning and layouts with a reinforcement learning formulation. We establish the core elements of our approach: agent, environment, action space and supporting components. Externally predicted workloads and the current physical design serve as input to our agent. The environment guides the search process by generating immediate rewards based on fresh cost estimates, for either the entirety or a sample of queries from the workload, and by deciding the possible actions given a state. This set of actions is configurable, enabling the representation of different partitioning problems. For use in an online setting the agent learns a fixed-length sequence of n actions that maximize the temporal reward for the predicted workload. Through an initial implementation we assert the feasibility of our approach. To conclude, we list open challenges for this work.
21
Reinforcement Learning: An Introduction
Richard S. Sutton, Andy Barto · IEEE Transactions on Neural Networks · 1998 · 27.1K citations
Kybernetes · 1998 · 1.6K citations
Carlo Curino, Evan P. C. Jones, Yang Zhang et al. · Proceedings of the VLDB Endowment · 2010 · 532 citations