Robo-Cortex: A Self-Evolving Embodied Agent via Dual-Grain Cognitive Memory and Autonomous Knowledge Induction

2026-05-18

AI Summary

This work addresses the limited navigation generalization of embodied agents in unknown environments caused by "experiential forgetting" by proposing a self-evolving navigation framework. The framework employs a reflect-adapt cycle that integrates an autonomous knowledge abstraction mechanism with a dual-granularity cognitive memory system—comprising short-term reflective memory and long-term principle memory—to distill multimodal trajectories into a structured repository of navigation heuristics. Strategy optimization is further enhanced through a multimodal imagination-verification loop and a vision-language model-based evaluator. Evaluated on the IGNav, AR, and AEQA benchmarks, the method significantly outperforms existing approaches, achieving up to a 4.16% absolute improvement in Success weighted by Path Length (SPL) and a 15.30% gain in cross-environment transfer settings. Real-world robotic experiments further validate its practical efficacy.

Abstract

The ability to navigate and interact with complex environments is central to real-world embodied agents, yet navigation in unseen environments remains challenging due to "experiential amnesia," where existing trajectory-driven or reactive policies fail to synthesize generalizable strategies from past interactions. We propose Robo-Cortex, a self-evolving framework that enables robots to autonomously induce navigation heuristics and refine cognitive strategies through a continuous reflection-adaptation loop. By abstracting success patterns and failure pitfalls into natural-language heuristics, Robo-Cortex enables a transition from passive execution to active strategy evolution. Our core innovation is an Autonomous Knowledge Induction (AKI) mechanism that distills multimodal trajectories into a structured Navigation Heuristic Library for knowledge generalization. The architecture further incorporates a Dual-Grain Cognitive Memory system, comprising a Short-term Reflective Memory (SRM) for real-time local progress analysis, and a Long-term Principle Memory (LPM) that abstracts past trajectories into reusable guiding and cautionary principles. To ensure robust decision-making, we introduce a multimodal Imagine-then-Verify loop, where a world model simulates potential outcomes and a VLM-based evaluator validates action plans. Extensive evaluations on IGNav, AR, and AEQA show that Robo-Cortex consistently outperforms strong baselines in both task success and exploration efficiency, with gains of up to +4.16% SPL over the strongest prior method and up to +15.30% SPL under heuristic transfer to unseen environments. Preliminary real-world robotic experiments further support the effectiveness of Robo-Cortex in physical settings.

Problem

• experiential amnesia
• embodied agents
• navigation in unseen environments
• generalizable strategies
• cognitive memory

Innovation

• Autonomous Knowledge Induction
• Dual-Grain Cognitive Memory
• Navigation Heuristic Library
• Imagine-then-Verify loop
• Self-Evolving Embodied Agent