A Calculus of Variations Approach to Stochastic Control

2025-09-01

AI Summary

This paper investigates necessary conditions for optimal Markovian control policies in finite-horizon stochastic control problems. Departing from conventional approaches based on dynamic programming or the stochastic maximum principle, it systematically extends classical calculus of variations to stochastic settings—integrating Itô calculus and stochastic analysis to rigorously derive first-order variational conditions for optimality, namely a stochastic Euler–Lagrange-type equation. Crucially, this framework does not require differentiability of the value function, thereby providing a novel necessity analysis tool for nonlinear and nonconvex stochastic control problems. As a key validation, the paper solves the classical Merton portfolio optimization problem analytically; the resulting explicit optimal policy coincides exactly with established results, confirming both the mathematical rigor and practical efficacy of the proposed method.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

We use classical tools from calculus of variations to formally derive necessary conditions for a Markov control to be optimal in a standard finite time horizon stochastic control problem. As an example, we solve the well-known Merton portfolio optimization problem.

Problem

• Deriving necessary conditions for optimal Markov control
• Using calculus of variations in stochastic control problems
• Solving Merton portfolio optimization as application example

Innovation

• Calculus of variations for stochastic control
• Necessary conditions for Markov optimality
• Solving Merton portfolio optimization problem