Agentic Router: An Execution-Grounded Continual Learning Approach With Memory

2026-08-10

AI Summary

This work addresses the challenge that large language models often generate invalid or high-risk commands in command-line network operations and lack mechanisms for continual learning grounded in real execution feedback. The authors propose a dual-path consequence-aware agent: one path generates multiple candidate actions, predicts their consequences, and re-ranks them based on utility and risk to improve selection quality; the other leverages SSH execution feedback to continually refine consequence prediction via LoRA fine-tuning and distills reusable experiences to broaden action coverage. By integrating real-world execution outcomes into continual learning without modifying the base model (Qwen3), this framework significantly enhances both actionable coverage and Top-1 execution success rate across multi-turn SONiC tasks, demonstrating the effectiveness and complementarity of the dual-path design.

Abstract

Large language model (LLM) agents provide a promising interface for command-line-based network operations, but a plausible command may still fail or introduce operational risk after execution. Existing approaches mainly focus on command generation or final configuration correctness, and do not use execution-grounded experience to jointly improve candidate coverage and action selection. We propose an execution-grounded dual-path consequence-aware agent for CLI-based SONiC operations, which generates multiple complete actions, predicts their execution consequences, and selects the final action through utility- and risk-aware reranking. The proposal-side path abstracts reusable operational lessons into retrievable guidance to improve feasible-action coverage without modifying the proposal LLM, while the selection-side path adapts the consequence predictor through session-level LoRA updates using real SSH feedback to improve conditional selection quality. Experiments over multi-turn SONiC operation sessions with different Qwen3 proposal models show that the framework improves feasible-action coverage and top-1 execution success, and that the two adaptation paths provide complementary gains over interaction.

Problem

• LLM agents
• execution grounding
• continual learning
• action selection
• operational risk

Innovation

• execution-grounded learning
• dual-path agent
• consequence-aware reranking
• LoRA adaptation
• retrievable operational memory