Rollback the World, Keep the Reflection: Rollback-Induced Reflection for Long-Horizon LLM Agents

2026-09-16

AI Summary

为解决长周期任务中单个错误动作导致的累积错误问题，提出了一种基于回滚诱导反思（RIR）的方法，通过选择性回滚和保留有用经验来指导后续决策。

Abstract

Large language model (LLM) agents increasingly tackle long-horizon tasks through multi-step environment interaction, yet a single erroneous action can alter subsequent states and observations, causing errors to compound over time. Existing methods either correct the context without repairing altered environment states or restore earlier states while discarding useful experience, making it difficult to both eliminate failure conditions and avoid repeating past mistakes. We argue that reliable recovery should instead be treated as a rollback-boundary control problem that jointly determines when to intervene, where to resume, and what information should survive recovery. Based on this view, we propose Rollback-Induced Reflection (RIR), a unified recovery framework that restores execution to a selected prior state while carrying forward reusable knowledge distilled from the abandoned trajectory to guide subsequent decisions. We further characterize recovery through a unified operator over rollback depth and retained memory, providing a general view of state restoration and knowledge retention. Experiments on three long-horizon benchmarks demonstrate that RIR consistently improves task performance across multiple LLM backbones, with structured reflection memory preserving useful experience and selective rollback enabling efficient recovery.

Problem

• long-horizon tasks
• multi-step environment interaction
• erroneous action
• altered states
• reliable recovery

Innovation

• Rollback-Induced Reflection
• reliable recovery
• state restoration
• knowledge retention
• long-horizon tasks