Should I Have Expressed a Different Intent? Counterfactual Generation for LLM-Based Autonomous Control

2026-01-27

AI Summary

This work addresses the challenge of understanding how varying expressions of user intent affect outcomes in large language model (LLM)-driven autonomous control systems. The authors propose the first counterfactual generation framework with formal reliability guarantees, modeling the closed-loop interaction among the user, the LLM agent, and the environment via a structural causal model. By integrating probabilistic inversion, test-time scaling, and conformal prediction, they construct a coverage-guaranteed conformal counterfactual mechanism. Evaluated on a wireless network control task, the method substantially outperforms re-execution baselines, producing counterfactual outcome sets that contain the true counterfactual result with high probability.

Technology Category

• Reasoning under Uncertainty: Causality
• Multiagent Systems: Adversarial Agents
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language model (LLM)-powered agents can translate high-level user intents into plans and actions in an environment. Yet after observing an outcome, users may wonder: What if I had phrased my intent differently? We introduce a framework that enables such counterfactual reasoning in agentic LLM-driven control scenarios, while providing formal reliability guarantees. Our approach models the closed-loop interaction between a user, an LLM-based agent, and an environment as a structural causal model (SCM), and leverages test-time scaling to generate multiple candidate counterfactual outcomes via probabilistic abduction. Through an offline calibration phase, the proposed conformal counterfactual generation (CCG) yields sets of counterfactual outcomes that are guaranteed to contain the true counterfactual outcome with high probability. We showcase the performance of CCG on a wireless network control use case, demonstrating significant advantages compared to naive re-execution baselines.

Problem

• counterfactual reasoning
• large language models
• autonomous control
• intent expression
• reliability guarantees

Innovation

• Conformal Counterfactual Generation
• Structural Causal Model
• Probabilistic Abduction
• Test-Time Scaling
• LLM-based Control