Formal Entropy-Regularized Control of Stochastic Systems

2026-03-05

AI Summary

Existing control methods for continuous-state stochastic systems lack formal guarantees while simultaneously accounting for entropy, making it difficult to balance predictability and performance. This work proposes the first extension of formal abstractions to entropy-regularized control. By constructing finite-state abstractions and deriving error bounds on the Kullback–Leibler (KL) divergence between continuous distributions and their discretizations relative to the uniform distribution, the approach enables joint optimization of trajectory distribution KL divergence and cumulative cost. The method supports the synthesis of provably correct entropy-aware controllers, demonstrating in multiple case studies its ability to preserve formal specifications of the original system while flexibly trading off control cost against system predictability.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Intelligent Robots: State Estimation
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Analyzing and controlling system entropy is a powerful tool for regulating predictability of control systems. Applications benefiting from such approaches range from reinforcement learning and data security to human-robot collaboration. In continuous-state stochastic systems, accurate entropy analysis and control remains a challenge. In recent years, finite-state abstractions of continuous systems have enabled control synthesis with formal performance guarantees on objectives such as stage costs. However, these results do not extend to entropy-based performance measures. We solve this problem by first obtaining bounds on the entropy of system discretizations using traditional formal-abstractions results, and then obtaining an additional bound on the difference between the entropy of a continuous distribution and that of its discretization. The resulting theory enables formal entropy-aware controller synthesis that trades predictability against control performance while preserving formal guarantees for the original continuous system. More specifically, we focus on minimizing the linear combination of the KL divergence of the system trajectory distribution to uniform -- our system entropy metric -- and a generic cumulative cost. We note that the bound we derive on the difference between the KL divergence to uniform of a given continuous distribution and its discretization can also be relevant in more general information-theoretic contexts. A set of case studies illustrates the effectiveness of the method.

Problem

• entropy-regularized control
• stochastic systems
• formal abstractions
• continuous-state systems
• KL divergence

Innovation

• entropy-regularized control
• formal abstractions
• stochastic systems
• KL divergence
• continuous-state systems