Catastrophic-risk-aware reinforcement learning with extreme-value-theory-based policy gradients

Parisa Davar; Frédéric Godin; Jose Garrido

Catastrophic-risk-aware reinforcement learning with extreme-value-theory-based policy gradients

Machine Learning 2024-07-01 v2 Risk Management

Authors: Parisa Davar , Frédéric Godin , Jose Garrido

Abstract

This paper tackles the problem of mitigating catastrophic risk (which is risk with very low frequency but very high severity) in the context of a sequential decision making process. This problem is particularly challenging due to the scarcity of observations in the far tail of the distribution of cumulative costs (negative rewards). A policy gradient algorithm is developed, that we call POTPG. It is based on approximations of the tail risk derived from extreme value theory. Numerical experiments highlight the out-performance of our method over common benchmarks, relying on the empirical distribution. An application to financial risk management, more precisely to the dynamic hedging of a financial option, is presented.

Keywords

reinforcement learning catastrophic risk extreme value theory policy gradient financial risk management

Cite

@article{arxiv.2406.15612,
  title  = {Catastrophic-risk-aware reinforcement learning with extreme-value-theory-based policy gradients},
  author = {Parisa Davar and Frédéric Godin and Jose Garrido},
  journal= {arXiv preprint arXiv:2406.15612},
  year   = {2024}
}

Comments

The Python code to replicate the various numerical experiments of this paper is available at https://github.com/parisadavar/EVT-policy-gradient-RL

Catastrophic-risk-aware reinforcement learning with extreme-value-theory-based policy gradients

Abstract

Keywords

Cite

Comments

Related papers