Reinforcement learning for trading systems and portfolios

Abstract

We propose to train trading systems by optimizing fi-nancial objective functions via reinforcement learning. The performance functions that we consider as value functions are profit or wealth, the Sharpe ratio and our recently proposed differential Sharpe ratio for on-line learning. In Moody & Wu (1997), we presented empirical results in controlled experiments that demon-strated the advantages of reinforcement learning rela-tive to supervised learning. Here we extend our pre-vious work to compare Q-Learning to a reinforcement learning technique based on real-time recurrent learn-ing (RTRL) that maximizes immediate reward. Our simulation results include a spectacular demon-stration of the presence of predictability in the monthly