Deep Direct Reinforcement Learning for Financial Signal Representation and Trading

TLDR

The paper investigates whether a computer can outperform experienced traders in financial asset trading, drawing inspiration from deep learning and reinforcement learning concepts. The study aims to develop a recurrent deep neural network for real‑time financial signal representation and trading, including a task‑aware backpropagation‑through‑time method to mitigate gradient vanishing. The system uses a recurrent deep neural network that first learns market‑condition features via deep learning, then applies reinforcement learning to make trading decisions, with a task‑aware backpropagation‑through‑time training scheme to address gradient vanishing. The neural system demonstrates robust performance on both stock and commodity futures markets across extensive testing scenarios.

Abstract

Can we train the computer to beat experienced traders for financial assert trading? In this paper, we try to address this challenge by introducing a recurrent deep neural network (NN) for real-time financial signal representation and trading. Our model is inspired by two biological-related learning concepts of deep learning (DL) and reinforcement learning (RL). In the framework, the DL part automatically senses the dynamic market condition for informative feature learning. Then, the RL module interacts with deep representations and makes trading decisions to accumulate the ultimate rewards in an unknown environment. The learning system is implemented in a complex NN that exhibits both the deep and recurrent structures. Hence, we propose a task-aware backpropagation through time method to cope with the gradient vanishing issue in deep training. The robustness of the neural system is verified on both the stock and the commodity future markets under broad testing conditions.