SciPhy Reinforcement Learning for Portfolio Optimization

2026-07-16

AI Summary

This study addresses the multi-period dynamic asset allocation problem faced by large institutional investors under trading costs and market impact. The authors propose a signal-driven robust optimization framework that formulates a continuous-time reinforcement learning model, innovatively projecting the Hamilton–Jacobi–Bellman (HJB) equation onto observed price paths and solving it offline in a single pass using physics-informed neural networks (PINNs), thereby circumventing traditional iterative procedures. The approach further incorporates a microstructure-based quadratic price impact model and a discrete target-holding mechanism tailored to short-term trading decisions. Empirical evaluation on a portfolio of 14 ETFs demonstrates that the method significantly improves out-of-sample Sharpe ratios while effectively controlling volatility and turnover, outperforming both static and myopic baseline strategies.

Abstract

This paper introduces a dynamic portfolio optimization framework for large institutional investors using Scientific Physics-Informed Reinforcement Learning (SciPhyRL). Formulated in continuous time over an extended state space that includes explicit cumulative costs, the approach leverages offline historical data to learn optimal, distribution-aware strategies. A core innovation reduces the optimization challenge to solving an HJB equation by projecting it onto observed trajectories as a pathwise Hamilton-Jacobi equation. This is solved directly from data using PINN in a single offline sweep, eliminating the need for traditional value or policy iteration. To make the method effective at practical short horizons, the control variable is recast from a continuous trading rate to a discrete target holding. This ensures signal-implied positions are reached immediately, while execution costs are evaluated against a microstructure-grounded quadratic price impact model. Evaluated on a $14$-asset ETF universe using an engineered oracle signal, the learned Gibbs policy yields substantial out-of-sample Sharpe ratio improvements over static and myopic baselines. The results demonstrate that the proposed framework successfully translates known signal quality into a robust, multi-period, and cost-aware allocation mechanism with strictly controlled volatility and turnover.

Problem

• Portfolio Optimization
• Reinforcement Learning
• Transaction Costs
• Price Impact
• Dynamic Allocation

Innovation

• Physics-Informed Reinforcement Learning
• Hamilton-Jacobi-Bellman equation
• Pathwise optimization
• PINN
• Portfolio optimization