No Time Like the Present: Agentic Test-Time Training for LLM Agents

2026-07-03

AI Summary

This work addresses the performance degradation of large language model agents in long-horizon tasks, which often stems from repetitive exploration, re-execution of failed actions, and strategy forgetting. To mitigate these issues, the authors propose Agentic Test-Time Training (aTTT), a continual test-time training framework tailored for multi-turn interactions. aTTT dynamically adapts model weights to evolving task demands and incorporates an n-gram-based token-level loss reweighting mechanism to suppress policy drift induced by self-training. Leveraging vLLM’s LoRA API, the method enables low-overhead online parameter updates. Experimental results demonstrate that aTTT improves success rates by 5.0 and 4.9 percentage points on ALFWorld and SWE-bench Lite, respectively, effectively alleviating capability degradation over extended task trajectories.

Abstract

LLM agents often degrade over long episodes: as trajectories grow, they revisit explored states, repeat failed actions, and lose strategies that previously worked. Test-time training (TTT) offers a way to adapt model weights to the evolving task state, but existing LLM TTT methods largely adapt once to a fixed input. We study continuous TTT in multi-turn agent episodes, where each update changes the policy that generates later training text. This creates a self-training loop that helps when new trajectory information appears, but can amplify drift when the agent gets stuck and repeatedly trains on similar text. We find that update-text repetition distinguishes these regimes and introduce Agentic Test-Time Training (aTTT), a token-level reweighting method that downweights the loss on tokens appearing in repeated $n$-grams from prior updates while leaving novel tokens fully weighted. To run such updates inside live episodes, we build a concurrent serving system using vLLM's runtime LoRA API, limiting overhead to 1.9$\times$ the no-TTT cost. aTTT improves success by up to 5.0 points on ALFWorld and 4.9 points on SWE-bench Lite. The gains concentrate where models already have task competence but drift over long trajectories, suggesting that aTTT mainly preserves existing competence rather than teaching new abilities.

Problem

• LLM agents
• test-time training
• strategy drift
• long-horizon tasks
• self-training loop

Innovation

• Agentic Test-Time Training
• token-level reweighting
• self-training loop
• repetition-aware adaptation
• concurrent LoRA serving