Proceedings of the AAAI Conference on Artificial Intelligence · 2020 · 159 citations · 16 references
BusinessFinanceFinancial ManagementFinancial MarketsMachine LearningPortfolio ManagementAsset Return ModelingFinancial ForecastingInvestmentReinforcement LearningAsset Information ModellingFinancial Time Series ForecastingMovement PredictionSequential OptimizationState Augmentation
Portfolio management seeks maximal profits or minimal risks by continuously extracting information from diverse data sources and optimizing decisions, making it a promising area for reinforcement learning. We propose SARL, a State-Augmented RL framework for portfolio management that tackles data heterogeneity and market uncertainty. SARL augments each asset’s state with a price‑movement prediction derived from financial data or news, enabling the RL agent to better handle heterogeneous inputs and non‑stationary markets. Experiments on Bitcoin and HighTech markets, using 7‑year Reuters news, show that SARL outperforms existing portfolio‑management methods in accumulated and risk‑adjusted profits, and simulations confirm that state augmentation significantly boosts performance.
Portfolio management (PM) is a fundamental financial planning task that aims to achieve investment goals such as maximal profits or minimal risks. Its decision process involves continuous derivation of valuable information from various data sources and sequential decision optimization, which is a prospective research direction for reinforcement learning (RL). In this paper, we propose SARL, a novel State-Augmented RL framework for PM. Our framework aims to address two unique challenges in financial PM: (1) data heterogeneity – the collected information for each asset is usually diverse, noisy and imbalanced (e.g., news articles); and (2) environment uncertainty – the financial market is versatile and non-stationary. To incorporate heterogeneous data and enhance robustness against environment uncertainty, our SARL augments the asset information with their price movement prediction as additional states, where the prediction can be solely based on financial data (e.g., asset prices) or derived from alternative sources such as news. Experiments on two real-world datasets, (i) Bitcoin market and (ii) HighTech stock market with 7-year Reuters news articles, validate the effectiveness of SARL over existing PM approaches, both in terms of accumulated profits and risk-adjusted profits. Moreover, extensive simulations are conducted to demonstrate the importance of our proposed state augmentation, providing new insights and boosting performance significantly over standard RL-based PM method and other baselines.
16
Distributed Representations of Words and Phrases and their Compositionality
Tomáš Mikolov, Ilya Sutskever, Kai Chen et al. · arXiv (Cornell University) · 2013 · 18.1K citations · Full text
Continuous control with deep reinforcement learning
Timothy Lillicrap, Jonathan J. Hunt, Alexander Pritzel et al. · arXiv (Cornell University) · 2016 · 6.8K citations · Full text
Computing, Computer Science, Physics +17