Control when confidence is costly

2024-06-20

arXiv.org

AI Summary

This paper addresses the joint optimization of utility maximization and resource constraints—including motion energy consumption and internal inference overhead—under partial observability. We propose a computation-cost-aware stochastic control framework: modeling inference accuracy cost as the relative entropy of state estimation and embedding it into the linear-quadratic-Gaussian (LQG) control objective, thereby unifying utility, action effort, and inference cost in a single optimization criterion. We first reveal a phase transition between inference accuracy and optimal policy in control, identifying a rational “act more to think less” trade-off and constructing a rotationally equivalent family of suboptimal inference strategies. We theoretically prove the existence of this accuracy–action coupling phase transition and show that global utility optimality is preserved under computational constraints. The framework provides a scalable, rational computational paradigm applicable to biological perceptual decision-making and energy-efficient AI systems.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We develop a version of stochastic control that accounts for computational costs of inference. Past studies identified efficient coding without control, or efficient control that neglects the cost of synthesizing information. Here we combine these concepts into a framework where agents rationally approximate inference for efficient control. Specifically, we study Linear Quadratic Gaussian (LQG) control with an added internal cost on the relative precision of the posterior probability over the world state. This creates a trade-off: an agent can obtain more utility overall by sacrificing some task performance, if doing so saves enough bits during inference. We discover that the rational strategy that solves the joint inference and control problem goes through phase transitions depending on the task demands, switching from a costly but optimal inference to a family of suboptimal inferences related by rotation transformations, each misestimate the stability of the world. In all cases, the agent moves more to think less. This work provides a foundation for a new type of rational computations that could be used by both brains and machines for efficient but computationally constrained control.

Problem

• Optimizing resource efficiency in partially observable environments
• Balancing utility maximization with internal computation costs
• Developing frugal inference strategies for control under uncertainty

Innovation

• POMDP framework optimizing inference as resource
• Phase transition from Bayes-optimal to frugal inference
• Structured family of strategies for adaptation