Authors:
(1) Xiao-Yang Liu, Hongyang Yang, Columbia University (xl2427,hy2500@columbia.edu);
(2) Jiechao Gao, University of Virginia (jg5ycn@virginia.edu);
(3) Christina Dan Wang (Corresponding Author), New York University Shanghai (christina.wang@nyu.edu). Table of Links Abstract and 1 Introduction 2 Related Works and 2.1 Deep Reinforcement Learning Algorithms 2.2 Deep Reinforcement Learning Libraries and 2.3 Deep Reinforcement Learning in Finance 3 The Proposed FinRL Framework and 3.1 Overview of FinRL Framework 3.2 Application Layer 3.3 Agent Layer 3.4 Environment Layer 3.5 Training-Testing-Trading Pipeline 4 Hands-on Tutorials and Benchmark Performance and 4.1 Backtesting Module 4.2 Baseline Strategies and Trading Metrics 4.3 Hands-on Tutorials 4.4 Use Case I: Stock Trading 4.5 Use Case II: Portfolio Allocation and 4.6 Use Case III: Cryptocurrencies Trading 5 Ecosystem of FinRL and Conclusions, and References 4.3 Hands-on Tutorials We provide tutorials to help users walk through the strategy design pipeline, i.e., get familiar with the stat-action-reward specifications in Table 1 and the agent-environment interactions in Fig. 1. Tutorial 1: Stock trading First, users specify the state at the application layer, i.e., the number of stocks, technical indicators, the initial capital, etc. Second, users provide start/end dates for training/testing periods, set the time granularity. FinRL instantiates an environment for the task, while the operations are transparent to users. FinRL uses standard APIs to download data and obtains a Pandas DataFrame containing the open-high-low-close prices and volume (OHLCV) data. FinRL preprocesses the OHLCV data by filling missing data and calculates technical indicators that are passed into the state. Third, users select a DRL library and a DRL algorithm. FinRL has default hyperparameters for daily stock trading task. During the testing period, users can tune these parameters to improve the trading performance. Finally, FinRL feeds time series data of the portfolio value into a backtesting module to plot charts. Please see examples in Section 4.4 and Section 4.6. Tutorial 2: Analyzing Trading Performance Before deploying a trading strategy, users need to fully evaluate its trading performance via backtesting. The trading performance can be easily evaluated using the automatic backtesting module in Section 4.1. The commonly used trading metrics and baseline strategies are given in Section 4.2. Cumulative return and Sharpe ratio are widely used metrics to evaluate overall performance of trading strategies. To gain more details about the strategy, the distribution of returns over the testing period and annualized return are provided to examine if the return is stable and consistent. Annualized volatility and maximum drawdown measure the robustness. This paper is available on arxiv under CC BY 4.0 DEED license. Authors: (1) Xiao-Yang Liu, Hongyang Yang, Columbia University (xl2427,hy2500@columbia.edu); (2) Jiechao Gao, University of Virginia (jg5ycn@virginia.edu); (3) Christina Dan Wang (Corresponding Author), New York University Shanghai (christina.wang@nyu.edu). Authors: Authors: (1) Xiao-Yang Liu, Hongyang Yang, Columbia University (xl2427,hy2500@columbia.edu); (2) Jiechao Gao, University of Virginia (jg5ycn@virginia.edu); (3) Christina Dan Wang ( Corresponding Author ), New York University Shanghai (christina.wang@nyu.edu). Corresponding Author Table of Links Abstract and 1 Introduction Abstract and 1 Introduction 2 Related Works and 2.1 Deep Reinforcement Learning Algorithms 2 Related Works and 2.1 Deep Reinforcement Learning Algorithms 2.2 Deep Reinforcement Learning Libraries and 2.3 Deep Reinforcement Learning in Finance 2.2 Deep Reinforcement Learning Libraries and 2.3 Deep Reinforcement Learning in Finance 3 The Proposed FinRL Framework and 3.1 Overview of FinRL Framework 3 The Proposed FinRL Framework and 3.1 Overview of FinRL Framework 3.2 Application Layer 3.2 Application Layer 3.3 Agent Layer 3.3 Agent Layer 3.4 Environment Layer 3.4 Environment Layer 3.5 Training-Testing-Trading Pipeline 3.5 Training-Testing-Trading Pipeline 4 Hands-on Tutorials and Benchmark Performance and 4.1 Backtesting Module 4 Hands-on Tutorials and Benchmark Performance and 4.1 Backtesting Module 4.2 Baseline Strategies and Trading Metrics 4.2 Baseline Strategies and Trading Metrics 4.3 Hands-on Tutorials 4.3 Hands-on Tutorials 4.4 Use Case I: Stock Trading 4.4 Use Case I: Stock Trading 4.5 Use Case II: Portfolio Allocation and 4.6 Use Case III: Cryptocurrencies Trading 4.5 Use Case II: Portfolio Allocation and 4.6 Use Case III: Cryptocurrencies Trading 5 Ecosystem of FinRL and Conclusions, and References 5 Ecosystem of FinRL and Conclusions, and References 4.3 Hands-on Tutorials We provide tutorials to help users walk through the strategy design pipeline, i.e., get familiar with the stat-action-reward specifications in Table 1 and the agent-environment interactions in Fig. 1. Tutorial 1: Stock trading Tutorial 1: Stock trading First, users specify the state at the application layer, i.e., the number of stocks, technical indicators, the initial capital, etc. Second, users provide start/end dates for training/testing periods, set the time granularity. FinRL instantiates an environment for the task, while the operations are transparent to users. FinRL uses standard APIs to download data and obtains a Pandas DataFrame containing the open-high-low-close prices and volume (OHLCV) data. FinRL preprocesses the OHLCV data by filling missing data and calculates technical indicators that are passed into the state. Third, users select a DRL library and a DRL algorithm. FinRL has default hyperparameters for daily stock trading task. During the testing period, users can tune these parameters to improve the trading performance. Finally, FinRL feeds time series data of the portfolio value into a backtesting module to plot charts. Please see examples in Section 4.4 and Section 4.6. Tutorial 2: Analyzing Trading Performance Tutorial 2: Analyzing Trading Performance Before deploying a trading strategy, users need to fully evaluate its trading performance via backtesting. The trading performance can be easily evaluated using the automatic backtesting module in Section 4.1. The commonly used trading metrics and baseline strategies are given in Section 4.2. Cumulative return and Sharpe ratio are widely used metrics to evaluate overall performance of trading strategies. To gain more details about the strategy, the distribution of returns over the testing period and annualized return are provided to examine if the return is stable and consistent. Annualized volatility and maximum drawdown measure the robustness. This paper is available on arxiv under CC BY 4.0 DEED license. This paper is available on arxiv under CC BY 4.0 DEED license. available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Unlock the Secrets of Algorithmic Trading

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

AI That Learns and Unlearns: The Exceptionally Smart EXPLORER

Fraud Detection to Robo Investing: AI in Finance is Rising

From AI-Powered Trading To Regulation and Compliance: What Does 2024 Look Like for Investment Tech?

The Intersection of AI and Finance Will Create New Careers

Four Use-Cases for AI Automation in 2024

AI That Learns and Unlearns: The Exceptionally Smart EXPLORER

Fraud Detection to Robo Investing: AI in Finance is Rising

From AI-Powered Trading To Regulation and Compliance: What Does 2024 Look Like for Investment Tech?

The Intersection of AI and Finance Will Create New Careers

Four Use-Cases for AI Automation in 2024

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps