Time-Inhomogeneous Volatility Aversion for Financial Applications of Reinforcement Learning

2026-02-12

AI Summary

This work addresses the limitation of traditional reinforcement learning, which optimizes only the expected cumulative return and fails to capture the nuanced trade-offs between temporal distribution and risk inherent in financial decision-making. To this end, we propose a novel time-inhomogeneous risk-sensitive reinforcement learning framework that enables flexible control over the temporal structure of returns by specifying target returns at arbitrary time steps and penalizing the uncertainty of individual-step rewards. Unlike existing approaches that apply risk measures solely to the overall return, our method introduces a customizable volatility-aversion mechanism, better aligning with real-world financial objectives. Empirical evaluations across multiple toy environments demonstrate the effectiveness and advantages of the proposed approach in shaping the temporal distribution of returns.

Technology Category

• Machine Learning: Reinforcement Learning
• Reasoning under Uncertainty: Sequential Decision Making
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In finance, sequential decision problems are often faced, for which reinforcement learning (RL) emerges as a promising tool for optimisation without the need of analytical tractability. However, the objective of classical RL is the expected cumulated reward, while financial applications typically require a trade-off between return and risk. In this work, we focus on settings where one cares about the time split of the total return, ruling out most risk-aware generalisations of RL which optimise a risk measure defined on the latter. We notice that a preference for homogeneous splits, which we found satisfactory for hedging, can be unfit for other problems, and therefore propose a new risk metric which still penalises uncertainty of the single rewards, but allows for an arbitrary planning of their target levels. We study the properties of the resulting objective and the generalisation of learning algorithms to optimise it. Finally, we show numerical results on toy examples.

Problem

• reinforcement learning
• financial applications
• risk-aware optimization
• time-inhomogeneous returns
• volatility aversion

Innovation

• time-inhomogeneous volatility aversion
• risk-aware reinforcement learning
• temporal return distribution
• financial sequential decision-making
• customizable reward targets