Mark your calendar for the exciting HONOR X Series launch on September 26! Unveiling the powerful X6a, X5 Plus, and Pad X9 with budget-friendly brilliance. COMS360 wins Best PR Agency in Southeast Asia, and our PAGEONE Group Chairman and CEO, Ron F. Jabal ...

Smart Investment Platform Binomo Enters The Philippine Market

Bangalore (Karnataka) [India], September 21: In 2023 Binomo was honored with two prestigious international awards at the WORLD BUSINESS OUTLOOK AWARDS and International Business Magazine Awards for “Best Trading Platform, Global 2023” and “Most Trust ...

Binomo Awarded the Most Reliable Trading Platform of 2023

Bangalore, Karnataka, India: In 2023 Binomo was honored with two prestigious international awards at the WORLD BUSINESS OUTLOOK AWARDS and International Business Magazine Awards for "Best Trading ...

Bangalore (Karnataka) [India], September 21: In 2023 Binomo was honored with two prestigious international awards at the WORLD BUSINESS OUTLOOK AWARDS and International Business Magazine Awards for "Best Trading Platform, Global 2023" and "Most Trusted ...

Maven MS Plus Size India International Beauty Pageant Season 6, 2023, was organized in Dwarka, Delhi at the Taj- Vivanta Hotel

A document verification service, for secure deposits and withdrawals of funds; A password validation system that helps the user create a strong password; Two-factor authentication (2FA) that prevents scammers from gaining access to the trader's personal ...

BusinessWire India Bangalore (Karnataka) [India], September 21: In 2023 Binomo was honored with two prestigious international awards at the WORLD BUSINESS OUTLOOK AWARDS and International Business Magazine Awards for "Best Trading Platform, Global 2023 ...

In 2023 Binomo was honored with two prestigious international awards at the WORLD BUSINESS OUTLOOK AWARDS and International Business Magazine Awards for "Best Trading Platform, Global 2023" and "Most Trusted Platform, Global 2023".

From September 5th, trading enthusiasts from around the world will gather for an exhilarating showdown asBinomoproudly presents Trading Titans: Battle of the Best. This premium tournament, taking place from September 5th to 29th,

Binomo Presents Trading Titans: Battle of the Best - USD 100,000 Prize Pool Showdown

The Grand Rocket in 'Kanakia Silicon Valley' Unveiled, Rockey Boys: R Madhavan and Nambi Narayanan Sir cheer for the Rocket of another kind!

Binomo

binomo

How to Accurately Measure Binomial Proportions for Reliable Conversion Metrics

MATHEMATICAL MEMORIES: NEWTON'S BINOMIAL THEOREM

Nature + Biomimicry + Bitcoin: Mutually Beneficial Partnership

Naturaleza + Biomimetismo + Bitcoin: una alianza mutuamente beneficiosa

Luonto + Biomimikri + Bitcoin: molemminpuolisesti hyödyllinen kumppanuus

प्रकृति + बायोमिमिक्री + बिटकॉइन: पारस्परिक रूप से लाभकारी साझेदारी

Nature + Biomimicry + Bitcoin. փոխշահավետ գործընկերություն

自然 + バイオミミクリー + ビットコイン: 相互に利益のあるパートナーシップ

Nature + Biomimicry + Bitcoin: ორმხრივად მომგებიანი პარტნიორობა

ធម្មជាតិ + Biomimicry + Bitcoin៖ ភាពជាដៃគូដែលមានអត្ថប្រយោជន៍ទៅវិញទៅមក

Natiora + Biomimicry + Bitcoin: Fiaraha-miombon'antoka

Masikirwo + Biomimicry + Bitcoin: Kudyidzana Kunobatsira Kudyidzana

Природа + биомимикрија + биткоин: обострано корисно партнерство

இயற்கை + பயோமிமிக்ரி + பிட்காயின்: பரஸ்பர நன்மை பயக்கும் கூட்டாண்மை

Tabiat + Biomimicry + Bitcoin: O'zaro manfaatli hamkorlik

Table of Links Abstract


Introduction
Background
Reinforcement Learning
Similar Work


Methodology
DRLAgent Design


Training Procedures


Testing Procedures


Results


SABR Experiments


Conclusions


Appendix A


References CONCLUSIONS In summary, this article fills a void in the DRL hedging field, which primarily focuses on European options, by introducing the application of DRL agents to hedge American put options. The study utilizes the DDPG algorithm for this purpose. Notably, while the state variables— comprising the asset price, current holding, and time-to-maturity—conform to existing literature, a novelty of this research lies in a unique reward function. This reward function accounts for both the negative absolute difference between the option value and the underlying asset position value and incorporates a quadratic penalty for transaction costs. The first round of experiments uses GBM asset paths for both training and testing, and initial results show that under the presence of transaction costs, the DRL agent outperforms the BS Delta and binomial tree hedging strategies. Another preliminary result of this work is that when the hedging agent underestimates the volatility, the DRL agent once again outperforms the BS Delta and Binomial strategies. This article extends the findings of the GBM results by training DRL agents using asset paths generated through a stochastic volatility model. Stochastic volatility model parameters are calibrated to each symbol by using 80 options from eight symbols, featuring two maturities and five strikes. In contrast to the GBM approach, where a binomial tree interpolation provides option prices at each time step, option prices for stochastic volatility experiments are computed using a Chebyshev interpolation method, which eliminates the requirement to compute average payoffs from MC simulations up to maturity or early exercise at each training step. The outcomes reveal that the DRL agent not only outperforms the BS Delta strategy when tested with paths generated from the same stochastic volatility model used in training, but the DRL agent also exhibits superior hedging performance, on average, as compared to BS Delta when testing with the realized underlying assets between the option sale date and maturity. Finally, note that as the Chebyshev pricing method is model agnostic, this work contributes a general framework that may be extended to train DRL agents for any desired underlying asset process. These results imply that DRL agents may be employed in an empirical setting to effectively hedge American put options by calibrating the training process to listed option data. A recommended future direction for more optimal results would be to retrain the agent more frequently as new option data becomes available. Further, a future study that examines the effect of training hyperparameters is recommended to aid practitioners in starting to train their own DRL agents. Authors:
(1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada (reilly.pickard@mail.utoronto.ca);
(2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. Table of Links Abstract Introduction
Background
Reinforcement Learning
Similar Work Methodology
DRLAgent Design Training Procedures Testing Procedures Results SABR Experiments Conclusions Appendix A References Abstract Abstract Abstract Introduction Background Reinforcement Learning Similar Work Introduction Background Background Reinforcement Learning Reinforcement Learning Similar Work Similar Work Methodology DRLAgent Design Methodology Methodology DRLAgent Design DRLAgent Design Training Procedures Training Procedures Training Procedures Testing Procedures Testing Procedures Testing Procedures Results Results Results SABR Experiments SABR Experiments SABR Experiments Conclusions Conclusions Conclusions Appendix A Appendix A Appendix A References References References CONCLUSIONS In summary, this article fills a void in the DRL hedging field, which primarily focuses on European options, by introducing the application of DRL agents to hedge American put options. The study utilizes the DDPG algorithm for this purpose. Notably, while the state variables— comprising the asset price, current holding, and time-to-maturity—conform to existing literature, a novelty of this research lies in a unique reward function. This reward function accounts for both the negative absolute difference between the option value and the underlying asset position value and incorporates a quadratic penalty for transaction costs. The first round of experiments uses GBM asset paths for both training and testing, and initial results show that under the presence of transaction costs, the DRL agent outperforms the BS Delta and binomial tree hedging strategies. Another preliminary result of this work is that when the hedging agent underestimates the volatility, the DRL agent once again outperforms the BS Delta and Binomial strategies. This article extends the findings of the GBM results by training DRL agents using asset paths generated through a stochastic volatility model. Stochastic volatility model parameters are calibrated to each symbol by using 80 options from eight symbols, featuring two maturities and five strikes. In contrast to the GBM approach, where a binomial tree interpolation provides option prices at each time step, option prices for stochastic volatility experiments are computed using a Chebyshev interpolation method, which eliminates the requirement to compute average payoffs from MC simulations up to maturity or early exercise at each training step. The outcomes reveal that the DRL agent not only outperforms the BS Delta strategy when tested with paths generated from the same stochastic volatility model used in training, but the DRL agent also exhibits superior hedging performance, on average, as compared to BS Delta when testing with the realized underlying assets between the option sale date and maturity. Finally, note that as the Chebyshev pricing method is model agnostic, this work contributes a general framework that may be extended to train DRL agents for any desired underlying asset process. These results imply that DRL agents may be employed in an empirical setting to effectively hedge American put options by calibrating the training process to listed option data. A recommended future direction for more optimal results would be to retrain the agent more frequently as new option data becomes available. Further, a future study that examines the effect of training hyperparameters is recommended to aid practitioners in starting to train their own DRL agents. Authors: (1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada (reilly.pickard@mail.utoronto.ca); (2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. Authors: Authors: (1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada ( reilly.pickard@mail.utoronto.ca ); reilly.pickard@mail.utoronto.ca (2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. available on arxiv available on arxiv

Advancements in Deep Reinforcement Learning for Hedging American Put Options

Table of Links Abstract


Introduction
Background
Reinforcement Learning
Similar Work


Methodology
DRLAgent Design


Training Procedures


Testing Procedures


Results


SABR Experiments


Conclusions


Appendix A


References RESULTS This section presents and discusses the results of numerical experiments. As described in the methodology, a DRL agent is first trained using simulations of a GBM process, and the testing results of this GBM trained agent are presented first. Not only are GBM experiments conducted with and without transaction costs, the robustness of the DRL agent is assessed by conducting an experiment at a volatility level higher than what was seen by the DRL agent in training. Next, the real-world applicability is assessed by testing DRL agents trained with market-calibrated stochastic volatility models, noting that the first stochastic volatility test uses arbitrary parameters. Geometric Brownian Motion Experiment In the first experiment, a DRL agent is trained using GBM asset paths. Exhibits 3 and 4 show the resulting final P&L distributions when the transaction cost rates, λ, are 0 and 3%, respectively. Note that separate subplots are used for ease of comparison, and the DRL agent results are the same when compared to the BS Delta and binomial strategies in each subplot. Exhibit 5 summarizes the mean and standard deviations of the final P&L for DRL, BS Delta, and binomial hedging, and the highest (lowest) mean (standard deviation) is italicized and underlined for each transaction cost rate. Exhibits 6 and 7 show sample hedging actions for one asset path that finishes out of the money (OTM), and one asset path that crosses the early exercise boundary. A comparison of Exhibits 3 and 4 shows the impact of a cost conscious DRL agent. When there are no transaction costs, the DRL agent hedges effectively, producing a higher mean final P&L than the BS Delta method. This is an expected result, as the BS Delta method does not consider early exercise. However, the DRL agent does not achieve a higher mean final P&L than the binomial strategy, and the standard deviation of final P&L’s is highest for the DRL agent. However, when there are 3% transaction costs, the DRL agent achieves a higher mean final P&L and lower standard deviation than both the BS Delta and binomial strategies. Exhibits 5 and 6 aid in illustrating the DRL agent’s awareness of premature exercise risk while maintaining a focus on transaction costs. At early time steps, the DRL agent hedges similarly to the binomial strategy, and the BS Delta strategy does not consider early exercise and, therefore, has the lowest hedge position. While the three strategies converge when the threat of early exercise decreases and the asset price moves out of the money, as is in the case in Exhibit 6, the DRL agent tends to react more slowly to abrupt price movements. When there is a spike in the required asset position, the DRL agent tends to under-hedge relative to the other two strategies. Likewise, the DRL agent tends to over-hedge when there is a dip in the required asset position. This displays the cost-consciousness of the DRL agent and helps show why the DRL method outperforms the other two strategies when transaction costs are present. The second GBM experiment aims to display the robustness of the DRL agent. Call to mind that the DRL agent is trained with a volatility of 20%, the binomial strategy is computed with trees constructed with 20% volatility, and the BS Delta strategy uses a volatility of 20% to compute both the option price and Delta. As such, let this 20% volatility be labelled as 𝜎𝑎𝑔𝑒𝑛𝑡. Now consider a scenario wherein GBM asset paths evolve with a volatility of 24%. Moreover, let the option buyer use the correct volatility of 24% to compute an exercise boundary for exercise decisions. As such, this volatility of 24% is given the label 𝜎𝑏𝑢𝑦𝑒𝑟. In this scenario, not only are the hedging agents using an incorrect volatility to hedge the option, but the initial option price is computed with a lower volatility, and the option is therefore sold under value. Exhibits 8 through 10 summarize the results of this volatility experiment. The results of Exhibits 8 through 10 show that the DRL agent is more robust to an inaccurate volatility estimate than the BS Delta and binomial tree strategies. With no transaction costs, the DRL agent has a higher mean final P&L than the other two strategies, and a lower standard deviation than the binomial strategy. With transaction costs considered, the DRL agent once again outperforms both the BS Delta and the binomial strategies, achieving a higher mean final P&L, as well as a lower standard deviation. Authors:
(1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada (reilly.pickard@mail.utoronto.ca);
(2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. Table of Links Abstract Introduction
Background
Reinforcement Learning
Similar Work Methodology
DRLAgent Design Training Procedures Testing Procedures Results SABR Experiments Conclusions Appendix A References Abstract Abstract Abstract Introduction Background Reinforcement Learning Similar Work Introduction Background Background Reinforcement Learning Reinforcement Learning Similar Work Similar Work Methodology DRLAgent Design Methodology Methodology DRLAgent Design DRLAgent Design Training Procedures Training Procedures Training Procedures Testing Procedures Testing Procedures Testing Procedures Results Results Results SABR Experiments SABR Experiments SABR Experiments Conclusions Conclusions Conclusions Appendix A Appendix A Appendix A References References References RESULTS This section presents and discusses the results of numerical experiments. As described in the methodology, a DRL agent is first trained using simulations of a GBM process, and the testing results of this GBM trained agent are presented first. Not only are GBM experiments conducted with and without transaction costs, the robustness of the DRL agent is assessed by conducting an experiment at a volatility level higher than what was seen by the DRL agent in training. Next, the real-world applicability is assessed by testing DRL agents trained with market-calibrated stochastic volatility models, noting that the first stochastic volatility test uses arbitrary parameters. Geometric Brownian Motion Experiment Geometric Brownian Motion Experiment In the first experiment, a DRL agent is trained using GBM asset paths. Exhibits 3 and 4 show the resulting final P&L distributions when the transaction cost rates, λ, are 0 and 3%, respectively. Note that separate subplots are used for ease of comparison, and the DRL agent results are the same when compared to the BS Delta and binomial strategies in each subplot. Exhibit 5 summarizes the mean and standard deviations of the final P&L for DRL, BS Delta, and binomial hedging, and the highest (lowest) mean (standard deviation) is italicized and underlined for each transaction cost rate. Exhibits 6 and 7 show sample hedging actions for one asset path that finishes out of the money (OTM), and one asset path that crosses the early exercise boundary. A comparison of Exhibits 3 and 4 shows the impact of a cost conscious DRL agent. When there are no transaction costs, the DRL agent hedges effectively, producing a higher mean final P&L than the BS Delta method. This is an expected result, as the BS Delta method does not consider early exercise. However, the DRL agent does not achieve a higher mean final P&L than the binomial strategy, and the standard deviation of final P&L’s is highest for the DRL agent. However, when there are 3% transaction costs, the DRL agent achieves a higher mean final P&L and lower standard deviation than both the BS Delta and binomial strategies. Exhibits 5 and 6 aid in illustrating the DRL agent’s awareness of premature exercise risk while maintaining a focus on transaction costs. At early time steps, the DRL agent hedges similarly to the binomial strategy, and the BS Delta strategy does not consider early exercise and, therefore, has the lowest hedge position. While the three strategies converge when the threat of early exercise decreases and the asset price moves out of the money, as is in the case in Exhibit 6, the DRL agent tends to react more slowly to abrupt price movements. When there is a spike in the required asset position, the DRL agent tends to under-hedge relative to the other two strategies. Likewise, the DRL agent tends to over-hedge when there is a dip in the required asset position. This displays the cost-consciousness of the DRL agent and helps show why the DRL method outperforms the other two strategies when transaction costs are present. The second GBM experiment aims to display the robustness of the DRL agent. Call to mind that the DRL agent is trained with a volatility of 20%, the binomial strategy is computed with trees constructed with 20% volatility, and the BS Delta strategy uses a volatility of 20% to compute both the option price and Delta. As such, let this 20% volatility be labelled as 𝜎𝑎𝑔𝑒𝑛𝑡. Now consider a scenario wherein GBM asset paths evolve with a volatility of 24%. Moreover, let the option buyer use the correct volatility of 24% to compute an exercise boundary for exercise decisions. As such, this volatility of 24% is given the label 𝜎𝑏𝑢𝑦𝑒𝑟. In this scenario, not only are the hedging agents using an incorrect volatility to hedge the option, but the initial option price is computed with a lower volatility, and the option is therefore sold under value. Exhibits 8 through 10 summarize the results of this volatility experiment. The results of Exhibits 8 through 10 show that the DRL agent is more robust to an inaccurate volatility estimate than the BS Delta and binomial tree strategies. With no transaction costs, the DRL agent has a higher mean final P&L than the other two strategies, and a lower standard deviation than the binomial strategy. With transaction costs considered, the DRL agent once again outperforms both the BS Delta and the binomial strategies, achieving a higher mean final P&L, as well as a lower standard deviation. Authors: (1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada (reilly.pickard@mail.utoronto.ca); (2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. Authors: Authors: (1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada ( reilly.pickard@mail.utoronto.ca ); reilly.pickard@mail.utoronto.ca (2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. available on arxiv available on arxiv

Results of Deep Reinforcement Learning Agent Performance in Hedging American Put Options

Table of Links Abstract


Introduction
Background
Reinforcement Learning
Similar Work


Methodology
DRLAgent Design


Training Procedures


Testing Procedures


Results


SABR Experiments


Conclusions


Appendix A


References TESTING PROCEDURES Once a DRL agent is trained, a testing procedure is designed to assess the hedging performance of the agent. First, note that the final P&L of the hedging agent is computed by tracking a money-market account position, denoted as 𝐵, through each time step until maturity: Given this defined performance metric, the main requirements for agent testing are: 1. The generation of asset price data, which is required for the DRL agent state and the P&L calculations. 2. The American put option price at the initial time step, used in the P&L calculation. 3. The construction of an exercise boundary to be used by the counterparty. 4. The design of benchmark hedging strategies for comparison. Once again, this section will be grouped into GBM and stochastic volatility model experiments. GBM Experiments: Testing Procedure s in training, GBM tests use MC paths of the GBM process for testing. Moreover, given that an American option tree was computed for training, the tree is used to generate the initial option price in testing. Further, by indexing the nodes at which exercise is optimal, an exercise boundary to be used by the counterparty is constructed. Given that binomial trees are constructed for both the asset and option price processes, this study uses a binomial tree hedge as a testing benchmark for the DRL agent. At a given timestep, the binomial tree hedge for an American put option, which in this study is denoted 𝛽𝑡 , is given by where 𝑁(∗) is the cumulative normal distribution function, 𝑟 is the risk-free rate (5% for all experiments), and 𝐾 is the strike price (Hull 2012). All GBM tests are conducted using 10000 GBM sample paths with 100 rebalance points each, and the initial asset price and strike price are both set at $100. Stochastic Volatility Experiments: Testing Procedure Recall that to establish a baseline, the first stochastic volatility experiment trains the DRL agent using arbitrary model coefficients. The option maturity is set to one month, and the DRL agent trained with paths from said stochastic volatility model with arbitrary coefficients is tested across 10000 episodes with 21 rebalance steps each (daily rebalancing), noting that the initial volatility is 20%, and the initial asset price and strike price are set to $100. Upon completion of this experiment, the market calibrated DRL agents are tested by simulating 10000 asset paths using the respective calibrated model, the implied initial volatility, and the option maturity. The calibrated Chebyshev exercise boundary is used for determining counterparty decisions. Daily rebalancing is assumed, and therefore the number of rebalance points in each testing episode is the number of trading days between August 17th, 2023, and the option maturity date. To give insight into how a DRL agent trained with a market-calibrated stochastic volatility model will perform in practice, a final experiment in this study evaluates the performance of each of the 80 DRL agents on the true asset path that was realized between August 17th, 2023, and the maturity date. The price data between August 17th, 2023, and November 17th, 2023, for all 8 symbols used were retrieved from Yahoo finance and are listed in the Appendix. As for the design of a comparative benchmark strategy, only the BS Delta is used as a comparative benchmark as a binomial tree is not constructed for stochastic volatility experiments. The updated volatility at each time step is substituted into the BS Delta calculation. Authors:
(1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada (reilly.pickard@mail.utoronto.ca);
(2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. Table of Links Abstract Introduction
Background
Reinforcement Learning
Similar Work Methodology
DRLAgent Design Training Procedures Testing Procedures Results SABR Experiments Conclusions Appendix A References Abstract Abstract Abstract Introduction Background Reinforcement Learning Similar Work Introduction Background Background Reinforcement Learning Reinforcement Learning Similar Work Similar Work Methodology DRLAgent Design Methodology Methodology DRLAgent Design DRLAgent Design Training Procedures Training Procedures Training Procedures Testing Procedures Testing Procedures Testing Procedures Results Results Results SABR Experiments SABR Experiments SABR Experiments Conclusions Conclusions Conclusions Appendix A Appendix A Appendix A References References References TESTING PROCEDURES Once a DRL agent is trained, a testing procedure is designed to assess the hedging performance of the agent. First, note that the final P&L of the hedging agent is computed by tracking a money-market account position, denoted as 𝐵, through each time step until maturity: Given this defined performance metric, the main requirements for agent testing are: 1. The generation of asset price data, which is required for the DRL agent state and the P&L calculations. 2. The American put option price at the initial time step, used in the P&L calculation. 3. The construction of an exercise boundary to be used by the counterparty. 4. The design of benchmark hedging strategies for comparison. Once again, this section will be grouped into GBM and stochastic volatility model experiments. GBM Experiments: Testing Procedure GBM Experiments: Testing Procedure s in training, GBM tests use MC paths of the GBM process for testing. Moreover, given that an American option tree was computed for training, the tree is used to generate the initial option price in testing. Further, by indexing the nodes at which exercise is optimal, an exercise boundary to be used by the counterparty is constructed. Given that binomial trees are constructed for both the asset and option price processes, this study uses a binomial tree hedge as a testing benchmark for the DRL agent. At a given timestep, the binomial tree hedge for an American put option, which in this study is denoted 𝛽𝑡 , is given by where 𝑁(∗) is the cumulative normal distribution function, 𝑟 is the risk-free rate (5% for all experiments), and 𝐾 is the strike price (Hull 2012). All GBM tests are conducted using 10000 GBM sample paths with 100 rebalance points each, and the initial asset price and strike price are both set at $100. Stochastic Volatility Experiments: Testing Procedure Stochastic Volatility Experiments: Testing Procedure Recall that to establish a baseline, the first stochastic volatility experiment trains the DRL agent using arbitrary model coefficients. The option maturity is set to one month, and the DRL agent trained with paths from said stochastic volatility model with arbitrary coefficients is tested across 10000 episodes with 21 rebalance steps each (daily rebalancing), noting that the initial volatility is 20%, and the initial asset price and strike price are set to $100. Upon completion of this experiment, the market calibrated DRL agents are tested by simulating 10000 asset paths using the respective calibrated model, the implied initial volatility, and the option maturity. The calibrated Chebyshev exercise boundary is used for determining counterparty decisions. Daily rebalancing is assumed, and therefore the number of rebalance points in each testing episode is the number of trading days between August 17th, 2023, and the option maturity date. To give insight into how a DRL agent trained with a market-calibrated stochastic volatility model will perform in practice, a final experiment in this study evaluates the performance of each of the 80 DRL agents on the true asset path that was realized between August 17th, 2023, and the maturity date. The price data between August 17th, 2023, and November 17th, 2023, for all 8 symbols used were retrieved from Yahoo finance and are listed in the Appendix. As for the design of a comparative benchmark strategy, only the BS Delta is used as a comparative benchmark as a binomial tree is not constructed for stochastic volatility experiments. The updated volatility at each time step is substituted into the BS Delta calculation. Authors: (1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada (reilly.pickard@mail.utoronto.ca); (2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. Authors: Authors: (1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada ( reilly.pickard@mail.utoronto.ca ); reilly.pickard@mail.utoronto.ca (2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. available on arxiv available on arxiv

Testing Procedures for Evaluating Deep Reinforcement Learning Agents in American Put Option Hedging

Table of Links Abstract


Introduction
Background
Reinforcement Learning
Similar Work


Methodology
DRLAgent Design


Training Procedures


Testing Procedures


Results


SABR Experiments


Conclusions


Appendix A


References TRAINING PROCEDURES Given the design of the state, action, and reward, training a DRL agent to hedge American put options requires the following: The generation of asset price data, used in the DRL agent state and reward.


The American put option price, used in the DRL agent reward. It is noted that counterparty exercise decisions are not considered in training, and training episodes continue past the expected exercise boundary. This is done to ensure that DRL agents experience prices below these boundaries, thereby improving robustness, as there is no guarantee that counterparties will exercise at the expected optimal boundary. While all DRL agents in this study have the same architecture and hyperparameters, the training procedure differs across two rounds of experiments. The first series of experiments uses MC simulations of a GBM process to generate asset price data, and a second series of experiments uses MC simulations of a calibrated stochastic volatility model as the underlying asset price process. GBM Experiments: Training Procedures In the GBM experiments, MC paths are simulated as: Stochastic Volatility Experiments: Training Procedures In the second round of experiments, MC paths of a stochastic volatility model are simulated. The stochastic volatility model setup is given by: As such, for the model used in this study a non-zero drift term 𝜇 is added, and the 𝛽 parameter is set equal to 1. Before calibrating the model to empirical option prices, a preliminary stochastic volatility experiment makes use of arbitrary model coefficients, setting 𝜌 to -0.4, which is reflective of the inverse relationship between asset prices and volatility (leverage effect) (Florescu and Pãsãricã 2009), and 𝜈 to 0.1. The stochastic volatility model is then calibrated to match true market assets. Specifically, given a listed price, maturity date, strike, and thereby an implied volatility, a DRL agent is implemented to hedge this specific option by using a calibrated stochastic volatility model for training. The empirical option data in this study is retrieved from the via the Bloomberg composite option monitor. For 8 symbols, put option prices on August 17th, 2023, for two different maturities (September 15th, 2023, and November 17th, 2023) and five different strikes were retrieved, giving a total of 80 options. Exhibit 1 summarizes the option data. Note that for each of the 80 options, the implied volatility is listed on the option monitor, and the option price is available by computing a mid-price of the bid-ask spread. For each option, model coefficients 𝜌 and 𝜈 are calibrated by setting the initial volatility to the retrieved implied volatility and minimizing the difference between the option mid-price and the average payoff given by 10000 model generated paths. Specifically, the constrained trustregion optimization algorithm is used. Given a parameter set for each symbol, the model parameters for each symbol are computed by taking an average. Note that the retrieved options are European, so no early exercise is considered when calibrating the model coefficients in this experiment. While the training and testing of the DRL agent considers American style options, model coefficients are calibrated without the exercise boundary, as the construction of said exercise boundary requires calibrated coefficients. As for option pricing at each time step, recall that GBM experiments use the interpolation of a binomial option tree. However, tree models increase in complexity when considering a stochastic volatility model with two random variables, as the tree now requires a second spatial dimension. Without a binomial tree, American put option prices in this study are computed by leveraging a technique presented by Glau, Mahlstedt, and Potz (2018) that involves approximating the value function at each time-step by a Chebyshev polynomial approximation. Chebyshev approximation involves fitting a series of orthogonal polynomials to a function (Chebyshev 1864). A main advantage to using Chebyshev pricing method for this work is the increased efficiency over a simulation based approach. For example, given an exercise boundary, such as one generated the popular Longstaff-Schwartz Monte Carlo (LSMC) (Longstaff and Schwartz 2001) method, option prices for a given asset price and timestep may be computed by simulating several thousand MC paths of the underlying process to expiry or exercise, and the option price is computed as the average payoff. However, in training an RL agent, simulation-based pricing would add a considerable amount of time to the process, as a new series of simulations is required for each step of training. In this study, with 5000 episodes of 25 steps each, 125,000 sets of simulations would be required, an unnecessary addition to the already time-consuming process of training a neural network. Moreover, it is noted that this Chebyshev method is agnostic to the underlying asset evolution process. Chebyshev interpolation involves weighing a set of orthogonal interpolating polynomials of increasing order so that their weighted sum approximates a known function between a predetermined upper and lower bound at each time step, which are determined from the extremal excursions of the underlying asset paths. The general Chebyshev process for pricing American options first requires discretizing the computational space from 𝑡0 = 0 to 𝑡𝑁 = 𝑇. Then, the following three-step process ensues: To further exemplify why this Chebyshev method was chosen over LSMC, the reader is directed to Glau, Mahlstedt, and Potz (2018) for a full performance comparison of the Chebyshev and LSMC exercise boundaries. While the Chebyshev method is used for stochastic volatility experiments, the accuracy of the Chebyshev pricing method may be assessed by plotting the Chebyshev exercise boundary for a GBM case where there is a known, true boundary generated by a binomial tree with 5000 nodes. Specifically, consider an at-the-money American put option struck at $100 with a 1-year maturity. Letting the underlying asset follow a GBM process, a comparison of the Chebyshev and binomial boundaries is given in Exhibit 2, and the results show a pronounced agreement between the two methods. Authors:
(1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada (reilly.pickard@mail.utoronto.ca);
(2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. Table of Links Abstract Introduction
Background
Reinforcement Learning
Similar Work Methodology
DRLAgent Design Training Procedures Testing Procedures Results SABR Experiments Conclusions Appendix A References Abstract Abstract Abstract Introduction Background Reinforcement Learning Similar Work Introduction Background Background Reinforcement Learning Reinforcement Learning Similar Work Similar Work Methodology DRLAgent Design Methodology Methodology DRLAgent Design DRLAgent Design Training Procedures Training Procedures Training Procedures Testing Procedures Testing Procedures Testing Procedures Results Results Results SABR Experiments SABR Experiments SABR Experiments Conclusions Conclusions Conclusions Appendix A Appendix A Appendix A References References References TRAINING PROCEDURES Given the design of the state, action, and reward, training a DRL agent to hedge American put options requires the following: The generation of asset price data, used in the DRL agent state and reward. The American put option price, used in the DRL agent reward. The generation of asset price data, used in the DRL agent state and reward. The generation of asset price data, used in the DRL agent state and reward. The American put option price, used in the DRL agent reward. The American put option price, used in the DRL agent reward. It is noted that counterparty exercise decisions are not considered in training, and training episodes continue past the expected exercise boundary. This is done to ensure that DRL agents experience prices below these boundaries, thereby improving robustness, as there is no guarantee that counterparties will exercise at the expected optimal boundary. While all DRL agents in this study have the same architecture and hyperparameters, the training procedure differs across two rounds of experiments. The first series of experiments uses MC simulations of a GBM process to generate asset price data, and a second series of experiments uses MC simulations of a calibrated stochastic volatility model as the underlying asset price process. GBM Experiments: Training Procedures GBM Experiments: Training Procedures In the GBM experiments, MC paths are simulated as: Stochastic Volatility Experiments: Training Procedures Stochastic Volatility Experiments: Training Procedures In the second round of experiments, MC paths of a stochastic volatility model are simulated. The stochastic volatility model setup is given by: As such, for the model used in this study a non-zero drift term 𝜇 is added, and the 𝛽 parameter is set equal to 1. Before calibrating the model to empirical option prices, a preliminary stochastic volatility experiment makes use of arbitrary model coefficients, setting 𝜌 to -0.4, which is reflective of the inverse relationship between asset prices and volatility (leverage effect) (Florescu and Pãsãricã 2009), and 𝜈 to 0.1. The stochastic volatility model is then calibrated to match true market assets. Specifically, given a listed price, maturity date, strike, and thereby an implied volatility, a DRL agent is implemented to hedge this specific option by using a calibrated stochastic volatility model for training. The empirical option data in this study is retrieved from the via the Bloomberg composite option monitor. For 8 symbols, put option prices on August 17th, 2023, for two different maturities (September 15th, 2023, and November 17th, 2023) and five different strikes were retrieved, giving a total of 80 options. Exhibit 1 summarizes the option data. Note that for each of the 80 options, the implied volatility is listed on the option monitor, and the option price is available by computing a mid-price of the bid-ask spread. For each option, model coefficients 𝜌 and 𝜈 are calibrated by setting the initial volatility to the retrieved implied volatility and minimizing the difference between the option mid-price and the average payoff given by 10000 model generated paths. Specifically, the constrained trustregion optimization algorithm is used. Given a parameter set for each symbol, the model parameters for each symbol are computed by taking an average. Note that the retrieved options are European, so no early exercise is considered when calibrating the model coefficients in this experiment. While the training and testing of the DRL agent considers American style options, model coefficients are calibrated without the exercise boundary, as the construction of said exercise boundary requires calibrated coefficients. As for option pricing at each time step, recall that GBM experiments use the interpolation of a binomial option tree. However, tree models increase in complexity when considering a stochastic volatility model with two random variables, as the tree now requires a second spatial dimension. Without a binomial tree, American put option prices in this study are computed by leveraging a technique presented by Glau, Mahlstedt, and Potz (2018) that involves approximating the value function at each time-step by a Chebyshev polynomial approximation. Chebyshev approximation involves fitting a series of orthogonal polynomials to a function (Chebyshev 1864). A main advantage to using Chebyshev pricing method for this work is the increased efficiency over a simulation based approach. For example, given an exercise boundary, such as one generated the popular Longstaff-Schwartz Monte Carlo (LSMC) (Longstaff and Schwartz 2001) method, option prices for a given asset price and timestep may be computed by simulating several thousand MC paths of the underlying process to expiry or exercise, and the option price is computed as the average payoff. However, in training an RL agent, simulation-based pricing would add a considerable amount of time to the process, as a new series of simulations is required for each step of training. In this study, with 5000 episodes of 25 steps each, 125,000 sets of simulations would be required, an unnecessary addition to the already time-consuming process of training a neural network. Moreover, it is noted that this Chebyshev method is agnostic to the underlying asset evolution process. Chebyshev interpolation involves weighing a set of orthogonal interpolating polynomials of increasing order so that their weighted sum approximates a known function between a predetermined upper and lower bound at each time step, which are determined from the extremal excursions of the underlying asset paths. The general Chebyshev process for pricing American options first requires discretizing the computational space from 𝑡0 = 0 to 𝑡𝑁 = 𝑇. Then, the following three-step process ensues: To further exemplify why this Chebyshev method was chosen over LSMC, the reader is directed to Glau, Mahlstedt, and Potz (2018) for a full performance comparison of the Chebyshev and LSMC exercise boundaries. While the Chebyshev method is used for stochastic volatility experiments, the accuracy of the Chebyshev pricing method may be assessed by plotting the Chebyshev exercise boundary for a GBM case where there is a known, true boundary generated by a binomial tree with 5000 nodes. Specifically, consider an at-the-money American put option struck at $100 with a 1-year maturity. Letting the underlying asset follow a GBM process, a comparison of the Chebyshev and binomial boundaries is given in Exhibit 2, and the results show a pronounced agreement between the two methods. Authors: (1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada (reilly.pickard@mail.utoronto.ca); (2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. Authors: Authors: (1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada ( reilly.pickard@mail.utoronto.ca ); reilly.pickard@mail.utoronto.ca (2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. available on arxiv available on arxiv

Hedging American Put Options with Deep Reinforcement Learning: Training Procedures

Table of Links Abstract


Introduction
Background
Reinforcement Learning
Similar Work


Methodology
DRLAgent Design


Training Procedures


Testing Procedures


Results


SABR Experiments


Conclusions


Appendix A


References Introduction BACKGROUND Following the sale of a financial option, traders seek to mitigate the associated risk through hedging strategies. For example, a trader that has just sold a European put option may hedge against adverse price drops by selling shares in the underlying asset. Traditionally, option hedgers attempt to maintain a Delta neutral portfolio, where Delta is the first partial derivative of the option price with respect to the underlying asset (Hull 2012). Therefore, the Delta neutral hedge for the European put option requires the sale of Delta shares of the underlying. For European options, analytic solutions for the option price and Delta exist via the Black and Scholes (BS) option pricing model (Black and Scholes 1973). While the BS model shows that the underlying risk of an option position is eliminated by a continuously rebalanced Delta-neutral portfolio, financial markets operate discretely in practice. Further, the BS model assumes constant volatility and no trading costs, which is not reflective of reality. As such, dynamic option hedging under market frictions is a progressive decision-making progress under uncertainty. One field that has garnered significant attention in dynamic decision-making procedures is reinforcement learning (RL), a subfield of artificial intelligence (AI). RL problems are aided in complex environments through neural network (NN) function approximation. The field concerning the combination of NN’s and RL is called deep RL (DRL), and DRL has been used to achieve super-human level performance in video games (Mnih et al. 2013), board games (Silver et al. 2016), and robot control (Lillicrap et al. 2015). As the dynamic hedging problem requires decision-making under uncertainty, several recent studies have used DRL to effectively hedge option positions. A review of 17 studies that use DRL for dynamic stock option hedging is given by Pickard and Lawryshyn (2023), who detail that while many studies show that DRL outperforms a Delta-neutral strategy when hedging European options under transaction costs and stochastic volatility, no current studies consider the hedging of American options. For American put options specifically, there is no analytical pricing or hedging formula available due to the potential of early exercise, and numerical methods for option pricing and hedging are required (Hull 2012). To address the literature gap pertaining to DRL agents that consider the potential for early option exercise, this article details the design of DRL agents that are trained to hedge American put options under transaction costs. DRL agents in this study are designed using the deep deterministic policy gradient (DDPG) method. In addition to training an American put DRL hedger when the underlying asset price follows a geometric Brownian motion (GBM), stochastic volatility is considered by calibrating stochastic volatility models using empirical option data on several stock symbols. Once these DRL agents are tested on simulated paths generated by the calibrated model, the hedging performance of each DRL agent is evaluated on the empirical asset price path for the respective symbol between the sale and maturity dates. Finally, note that DRL agent reward function requires the option price at each time step. While an interpolation of a binomial American option tree is used in GBM cases, this study employs the use of a Chebyshev interpolation method first proposed by Glau, Mahlstedt, and Potz (2018) for the determination of the option price in stochastic volatility experiments. This Chebyshev method is model-agnostic, and this work thereby provides a framework that extends seamlessly to more intricate processes. Moreover, the Chebyshev method allows the American option price to be computed more efficiently in stochastic volatility settings, as the requirement to average the payoff of several thousand Monte Carlo (MC) simulations from the current price level to expiry or exercise is eliminated. This Chebyshev pricing method is described in detail in the methodology section of this work. The rest of this section is dedicated to the introduction of DRL and a detailed account of similar work in the DRL hedging space. This article will then detail the methodology used to train DRL agents, before presenting and discussing the results of all numerical experiments. Authors:
(1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada (reilly.pickard@mail.utoronto.ca);
(2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada;
(5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. Table of Links Abstract Introduction
Background
Reinforcement Learning
Similar Work Methodology
DRLAgent Design Training Procedures Testing Procedures Results SABR Experiments Conclusions Appendix A References Abstract Abstract Abstract Introduction Background Reinforcement Learning Similar Work Introduction Background Background Reinforcement Learning Reinforcement Learning Similar Work Similar Work Methodology DRLAgent Design Methodology Methodology DRLAgent Design DRLAgent Design Training Procedures Training Procedures Training Procedures Testing Procedures Testing Procedures Testing Procedures Results Results Results SABR Experiments SABR Experiments SABR Experiments Conclusions Conclusions Conclusions Appendix A Appendix A Appendix A References References References Introduction BACKGROUND Following the sale of a financial option, traders seek to mitigate the associated risk through hedging strategies. For example, a trader that has just sold a European put option may hedge against adverse price drops by selling shares in the underlying asset. Traditionally, option hedgers attempt to maintain a Delta neutral portfolio, where Delta is the first partial derivative of the option price with respect to the underlying asset (Hull 2012). Therefore, the Delta neutral hedge for the European put option requires the sale of Delta shares of the underlying. For European options, analytic solutions for the option price and Delta exist via the Black and Scholes (BS) option pricing model (Black and Scholes 1973). While the BS model shows that the underlying risk of an option position is eliminated by a continuously rebalanced Delta-neutral portfolio, financial markets operate discretely in practice. Further, the BS model assumes constant volatility and no trading costs, which is not reflective of reality. As such, dynamic option hedging under market frictions is a progressive decision-making progress under uncertainty. One field that has garnered significant attention in dynamic decision-making procedures is reinforcement learning (RL), a subfield of artificial intelligence (AI). RL problems are aided in complex environments through neural network (NN) function approximation. The field concerning the combination of NN’s and RL is called deep RL (DRL), and DRL has been used to achieve super-human level performance in video games (Mnih et al. 2013), board games (Silver et al. 2016), and robot control (Lillicrap et al. 2015). As the dynamic hedging problem requires decision-making under uncertainty, several recent studies have used DRL to effectively hedge option positions. A review of 17 studies that use DRL for dynamic stock option hedging is given by Pickard and Lawryshyn (2023), who detail that while many studies show that DRL outperforms a Delta-neutral strategy when hedging European options under transaction costs and stochastic volatility, no current studies consider the hedging of American options. For American put options specifically, there is no analytical pricing or hedging formula available due to the potential of early exercise, and numerical methods for option pricing and hedging are required (Hull 2012). To address the literature gap pertaining to DRL agents that consider the potential for early option exercise, this article details the design of DRL agents that are trained to hedge American put options under transaction costs. DRL agents in this study are designed using the deep deterministic policy gradient (DDPG) method. In addition to training an American put DRL hedger when the underlying asset price follows a geometric Brownian motion (GBM), stochastic volatility is considered by calibrating stochastic volatility models using empirical option data on several stock symbols. Once these DRL agents are tested on simulated paths generated by the calibrated model, the hedging performance of each DRL agent is evaluated on the empirical asset price path for the respective symbol between the sale and maturity dates. Finally, note that DRL agent reward function requires the option price at each time step. While an interpolation of a binomial American option tree is used in GBM cases, this study employs the use of a Chebyshev interpolation method first proposed by Glau, Mahlstedt, and Potz (2018) for the determination of the option price in stochastic volatility experiments. This Chebyshev method is model-agnostic, and this work thereby provides a framework that extends seamlessly to more intricate processes. Moreover, the Chebyshev method allows the American option price to be computed more efficiently in stochastic volatility settings, as the requirement to average the payoff of several thousand Monte Carlo (MC) simulations from the current price level to expiry or exercise is eliminated. This Chebyshev pricing method is described in detail in the methodology section of this work. The rest of this section is dedicated to the introduction of DRL and a detailed account of similar work in the DRL hedging space. This article will then detail the methodology used to train DRL agents, before presenting and discussing the results of all numerical experiments. Authors: (1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada (reilly.pickard@mail.utoronto.ca); (2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. Authors: Authors: (1) Reilly Pickard, Department of Mechanical and Industrial Engineering, University of Toronto, Toronto, ON M5S 3G8, Canada ( reilly.pickard@mail.utoronto.ca ); reilly.pickard@mail.utoronto.ca (2) Finn Wredenhagen, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (3) Julio DeJesus, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (4) Mario Schlener, Ernst & Young LLP, Toronto, ON, M5H 0B3, Canada; (5) Yuri Lawryshyn, Department of Chemical Engineering and Applied Chemistry, University of Toronto, Toronto, ON M5S 3E5, Canada. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. This paper is available on arxiv under CC BY-NC-SA 4.0 Deed (Attribution-Noncommercial-Sharelike 4.0 International) license. available on arxiv available on arxiv

How Deep RL Enhances Hedging Strategies for American Put Options in Volatile Markets

Science: The Way of Nature

ሳይንስ፡ የተፈጥሮ መንገድ

Ciencia: Naturaleza ukan thakhipa

Наука: Пътят на природата

বিজ্ঞান: প্রকৃতির পথ

Ciencia: El camino de la naturaleza

Zientzia: Naturaren Bidea

מדע: דרך הטבע

科学：自然の道

ວິທະຍາສາດ: ວິທີການຂອງທໍາມະຊາດ

Nauka: Droga Natury

ساینس: د طبیعت لاره

Наука: Пут природе

Sports analytics has long relied on basic statistics such as points per game, field goal percentage, and rebounds to assess player performance. While these metrics provide some insight, they fall short of capturing the nuances of a player's skill and decision-making abilities. Coaches and analysts often struggle with evaluating the quality of a player's shots, which is crucial for devising effective game strategies. In that realm, Ricky Zhang has made strides. A data scientist at Amazon with a background in economics, mathematics, and computer science from Emory University, Zhang has developed a project that is set to redefine how we understand player performance on the basketball court. His 'Shot Quality' project, using advanced machine learning techniques, specifically beta-binomial regression, addresses the limitations of traditional basketball metrics. The Problem: Stale Metrics and Missed Opportunities Basketball analytics, for ages, leaned on basic stats to gauge player performance. These metrics, though handy, miss the finer points of a player’s skills and decisions. Coaches and analysts often hit walls when judging shot quality—a key factor for crafting game strategies. Ricky Zhang saw these gaps and aimed higher. With his machine learning and AI chops, he was inspired to craft a model that nails down the true quality of basketball shots. A New Era in Basketball Analytics Inspired by a baseball batting averages analytics report, Zhang started creating a model that could accurately assess the quality of shots taken by basketball players. The 'Shot Quality' project was born out of this vision. Zhang’s project utilizes beta-binomial regression, a statistical method that accounts for the variability in players' shooting performance and the trust placed in them by their coaches. The 'Shot Quality' project zeroes in on the context and conditions of player shots. By crunching heaps of data—player positioning, defensive pressure, game situations—Zhang's model predicts shot success likelihood. This fresh method paints a clearer picture of a player's shooting prowess than old-school metrics. One of the biggest challenges in sports analytics is the mountain of data and its tricky interpretation. Zhang's machine learning chops let him efficiently sift through this data, pulling out valuable insights. His model tackles data noise and variability, giving coaches and analysts straightforward, actionable information. The 'Shot Quality' project's impact? Massive. Coaches can now make smarter calls on player rotations, shot choices, and defensive matchups. Knowing which players excel in specific scenarios lets teams play to their strengths and exploit opponents' weaknesses. Ricky Zhang: A Thought Leader in Sports Analytics Zhang's work redefines industry standards. His innovative machine learning approach places him as a thought leader. Beyond technical skills, he's committed to mentoring upcoming data scientists and sharing his insights. The heart of his leadership is all about resilience and adaptability, evident in how he tackled personal hurdles. When Twitch let him go due to company-wide cutbacks, he swiftly pivoted, using his network and skills to land a new gig at Amazon. This journey bolstered his resilience and dedication to nonstop learning—key ingredients in his recipe for success. Zhang's influence in sports analytics is poised to grow. He envisions further refining the 'Shot Quality' model and expanding its applications to other sports. His goal is to integrate real-time data and advanced AI techniques to provide even more accurate and actionable insights. Recent leaps in sports analytics are shining a light on how critical data-driven decisions have become in athletics. Player-tracking tech and high-res video analysis are shaking up the game, capturing every little player's move and interaction on the court, creating a treasure trove of data for the geeks to dive into. The brains at MIT Sloan Sports Analytics Conference are making noise about how machine learning and AI are flipping sports strategies on their heads. Turns out, models that mix player tracking with game context data can boost the accuracy of performance predictions and strategic calls. Take neural networks, for instance—these bad boys are getting good at predicting player performance and spotting injury risks, giving teams new ways to keep their stars healthy and sharp. Then there’s wearable tech, adding another layer to the data game. Real-time physiological and biomechanical stats are now at coaches’ fingertips, helping keep tabs on player fatigue, recovery, and overall health. This means smarter coaching decisions and players who are primed for peak performance. Zhang's 'Shot Quality' project aligns with these trends, showcasing how advanced statistical models can be applied to sports analytics. By leveraging beta-binomial regression, Zhang's model accounts for the inherent variability in player performance, providing a more nuanced understanding of shooting efficiency. This approach not only enhances the accuracy of performance metrics but also offers deeper insights into player decision-making and game dynamics. The Road Ahead for Ricky Zhang and Shot Quality Project Ricky Zhang’s 'Shot Quality' project represents a significant leap forward in sports analytics. By applying advanced machine learning techniques to basketball performance, he is transforming how we understand and evaluate players. As Zhang continues to innovate and lead in this field, the future of sports analytics looks incredibly promising. For those inspired by Zhang’s journey, there are countless opportunities to explore the intersections of data science and sports. Whether you're an aspiring data scientist or a seasoned professional, Zhang's work serves as a powerful reminder of the impact that innovative thinking and perseverance can have on an industry. If you’re interested in learning more about Ricky Zhang’s work or exploring the future of sports analytics, connect with him on LinkedIn. Stay tuned for more groundbreaking projects that will continue to reshape the world of sports. Sports analytics has long relied on basic statistics such as points per game, field goal percentage, and rebounds to assess player performance. While these metrics provide some insight, they fall short of capturing the nuances of a player's skill and decision-making abilities. Coaches and analysts often struggle with evaluating the quality of a player's shots, which is crucial for devising effective game strategies. In that realm, Ricky Zhang has made strides. Ricky Zhang Ricky Zhang A data scientist at Amazon with a background in economics, mathematics, and computer science from Emory University, Zhang has developed a project that is set to redefine how we understand player performance on the basketball court. His 'Shot Quality' project, using advanced machine learning techniques, specifically beta-binomial regression, addresses the limitations of traditional basketball metrics. The Problem: Stale Metrics and Missed Opportunities Basketball analytics, for ages, leaned on basic stats to gauge player performance. These metrics, though handy, miss the finer points of a player’s skills and decisions. Coaches and analysts often hit walls when judging shot quality—a key factor for crafting game strategies. Ricky Zhang saw these gaps and aimed higher. With his machine learning and AI chops, he was inspired to craft a model that nails down the true quality of basketball shots. A New Era in Basketball Analytics Inspired by a baseball batting averages analytics report , Zhang started creating a model that could accurately assess the quality of shots taken by basketball players. The 'Shot Quality' project was born out of this vision. Zhang’s project utilizes beta-binomial regression, a statistical method that accounts for the variability in players' shooting performance and the trust placed in them by their coaches. baseball batting averages analytics report baseball batting averages analytics report The 'Shot Quality' project zeroes in on the context and conditions of player shots. By crunching heaps of data—player positioning, defensive pressure, game situations—Zhang's model predicts shot success likelihood. This fresh method paints a clearer picture of a player's shooting prowess than old-school metrics. One of the biggest challenges in sports analytics is the mountain of data and its tricky interpretation. Zhang's machine learning chops let him efficiently sift through this data, pulling out valuable insights. His model tackles data noise and variability, giving coaches and analysts straightforward, actionable information. The 'Shot Quality' project's impact? Massive. Coaches can now make smarter calls on player rotations, shot choices, and defensive matchups. Knowing which players excel in specific scenarios lets teams play to their strengths and exploit opponents' weaknesses. Ricky Zhang: A Thought Leader in Sports Analytics Zhang's work redefines industry standards. His innovative machine learning approach places him as a thought leader. Beyond technical skills, he's committed to mentoring upcoming data scientists and sharing his insights. The heart of his leadership is all about resilience and adaptability, evident in how he tackled personal hurdles. When Twitch let him go due to company-wide cutbacks, he swiftly pivoted, using his network and skills to land a new gig at Amazon. This journey bolstered his resilience and dedication to nonstop learning—key ingredients in his recipe for success. Zhang's influence in sports analytics is poised to grow. He envisions further refining the 'Shot Quality' model and expanding its applications to other sports. His goal is to integrate real-time data and advanced AI techniques to provide even more accurate and actionable insights. Recent leaps in sports analytics are shining a light on how critical data-driven decisions have become in athletics. Player-tracking tech and high-res video analysis are shaking up the game, capturing every little player's move and interaction on the court, creating a treasure trove of data for the geeks to dive into. The brains at MIT Sloan Sports Analytics Conference are making noise about how machine learning and AI are flipping sports strategies on their heads. Turns out, models that mix player tracking with game context data can boost the accuracy of performance predictions and strategic calls. Take neural networks, for instance—these bad boys are getting good at predicting player performance and spotting injury risks, giving teams new ways to keep their stars healthy and sharp. Then there’s wearable tech, adding another layer to the data game. Real-time physiological and biomechanical stats are now at coaches’ fingertips, helping keep tabs on player fatigue, recovery, and overall health. This means smarter coaching decisions and players who are primed for peak performance. Zhang's 'Shot Quality' project aligns with these trends, showcasing how advanced statistical models can be applied to sports analytics. By leveraging beta-binomial regression, Zhang's model accounts for the inherent variability in player performance, providing a more nuanced understanding of shooting efficiency. This approach not only enhances the accuracy of performance metrics but also offers deeper insights into player decision-making and game dynamics. The Road Ahead for Ricky Zhang and Shot Quality Project Ricky Zhang’s 'Shot Quality' project represents a significant leap forward in sports analytics. By applying advanced machine learning techniques to basketball performance, he is transforming how we understand and evaluate players. As Zhang continues to innovate and lead in this field, the future of sports analytics looks incredibly promising. For those inspired by Zhang’s journey, there are countless opportunities to explore the intersections of data science and sports. Whether you're an aspiring data scientist or a seasoned professional, Zhang's work serves as a powerful reminder of the impact that innovative thinking and perseverance can have on an industry. If you’re interested in learning more about Ricky Zhang’s work or exploring the future of sports analytics, connect with him on LinkedIn . Stay tuned for more groundbreaking projects that will continue to reshape the world of sports. LinkedIn LinkedIn

Stories, News & Mentions about BINOMO

Binomo’s Stories on HackerNoon

Binomo's News & Mentions