Learning to Wait: Synchronizing Agents with the Physical World

2025-12-18

AI Summary

Real-world agents face a cognition–physics temporal misalignment problem caused by action asynchrony: non-blocking actions inherently exhibit latency between initiation and completion, while existing environment-side solutions—such as blocking wrappers or high-frequency polling—sacrifice scalability or dilute contextual coherence. This paper introduces a novel agent-side temporal alignment paradigm, extending the “Code-as-Action” framework to the time dimension for the first time. It enables large language models to internalize “time-awareness” via semantic priors and in-context learning (ICL). At its core, the agent proactively predicts wait durations, enabling semantic-driven, adaptive temporal control. Evaluated in a Kubernetes simulation environment, our approach significantly reduces query overhead and execution latency. Results demonstrate that time awareness is not only learnable but also critical for autonomous adaptation and evolution in open-ended environments.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Agent Communication
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Real-world agentic tasks, unlike synchronous Markov Decision Processes (MDPs), often involve non-blocking actions with variable latencies, creating a fundamental extit{Temporal Gap} between action initiation and completion. Existing environment-side solutions, such as blocking wrappers or frequent polling, either limit scalability or dilute the agent's context window with redundant observations. In this work, we propose an extbf{Agent-side Approach} that empowers Large Language Models (LLMs) to actively align their extit{Cognitive Timeline} with the physical world. By extending the Code-as-Action paradigm to the temporal domain, agents utilize semantic priors and In-Context Learning (ICL) to predict precise waiting durations ( exttt{time.sleep(t)}), effectively synchronizing with asynchronous environment without exhaustive checking. Experiments in a simulated Kubernetes cluster demonstrate that agents can precisely calibrate their internal clocks to minimize both query overhead and execution latency, validating that temporal awareness is a learnable capability essential for autonomous evolution in open-ended environments.

Problem

• Agents struggle with variable action latencies in real-world asynchronous environments
• Existing solutions limit scalability or overload agents with redundant observations
• LLMs lack temporal awareness to synchronize cognitive timelines with physical world

Innovation

• Agent-side approach synchronizes LLMs with physical world timelines
• Extends Code-as-Action paradigm to predict waiting durations via ICL
• Minimizes query overhead and latency by calibrating internal clocks