Unlock the Secrets of Algorithmic Trading

Table of Links

Abstract and 1 Introduction

2 Related Works and 2.1 Deep Reinforcement Learning Algorithms

2.2 Deep Reinforcement Learning Libraries and 2.3 Deep Reinforcement Learning in Finance

3 The Proposed FinRL Framework and 3.1 Overview of FinRL Framework

3.2 Application Layer

3.3 Agent Layer

3.4 Environment Layer

3.5 Training-Testing-Trading Pipeline

4 Hands-on Tutorials and Benchmark Performance and 4.1 Backtesting Module

4.2 Baseline Strategies and Trading Metrics

4.3 Hands-on Tutorials

4.4 Use Case I: Stock Trading

4.5 Use Case II: Portfolio Allocation and 4.6 Use Case III: Cryptocurrencies Trading

5 Ecosystem of FinRL and Conclusions, and References

4.3 Hands-on Tutorials

We provide tutorials to help users walk through the strategy design pipeline, i.e., get familiar with the stat-action-reward specifications in Table 1 and the agent-environment interactions in Fig. 1.

Tutorial 1: Stock trading

First, users specify the state at the application layer, i.e., the number of stocks, technical indicators, the initial capital, etc. Second, users provide start/end dates for training/testing periods, set the time granularity. FinRL instantiates an environment for the task, while the operations are transparent to users. FinRL uses standard APIs to download data and obtains a Pandas DataFrame containing the open-high-low-close prices and volume (OHLCV) data. FinRL preprocesses the OHLCV data by filling missing data and calculates technical indicators that are passed into the state. Third, users select a DRL library and a DRL algorithm. FinRL has default hyperparameters for daily stock trading task. During the testing period, users can tune these parameters to improve the trading performance. Finally, FinRL feeds time series data of the portfolio value into a backtesting module to plot charts. Please see examples in Section 4.4 and Section 4.6.

Tutorial 2: Analyzing Trading Performance

Before deploying a trading strategy, users need to fully evaluate its trading performance via backtesting. The trading performance can be easily evaluated using the automatic backtesting module in Section 4.1. The commonly used trading metrics and baseline strategies are given in Section 4.2.

Cumulative return and Sharpe ratio are widely used metrics to evaluate overall performance of trading strategies. To gain more details about the strategy, the distribution of returns over the testing period and annualized return are provided to examine if the return is stable and consistent. Annualized volatility and maximum drawdown measure the robustness.