RAVEN: Long-Horizon Reasoning & Navigation with a Visuo-Spatio-Temporal Memory

2026-06-23

AI Summary

This work addresses the absence of memory systems in long-term robot deployments that simultaneously achieve compactness, scalability, and fine-grained visual-semantic understanding with precise spatiotemporal grounding. The authors propose an intelligent memory architecture tailored for long-horizon question answering and navigation, which eschews conventional image captioning and instead directly stores pose- and timestamp-annotated visual embeddings in a vector database, coupled with a spatial map for grounded retrieval. This approach fully preserves semantic, spatial, and temporal information while substantially reducing retrieval overhead. Experiments demonstrate that the system matches or exceeds the performance of state-of-the-art vision-language models on multiple simulated and real-world video question-answering benchmarks—at only one-tenth of the retrieval cost—and has been successfully deployed on a Unitree Go1 robot to enable natural language navigation in large-scale indoor environments.

Abstract

Long-term robot deployment requires a compact and scalable memory that preserves fine-grained visual semantics, grounds observations in space and time, and enables efficient storage and retrieval. In this paper, we propose RAVEN, an agentic memory system for long-horizon robotic question answering and navigation. RAVEN stores visual embeddings with pose and time in a vector database, and grounds retrieval in a spatial map to answer queries and navigate to goals. By operating directly on visual embeddings, RAVEN avoids lossy image-to-text captioning and enables accurate semantic, spatial, and temporal retrieval at scale. Across several simulated and real-world video question-answering benchmarks, RAVEN consistently surpasses caption-based memory systems and matches frontier VLMs on long-horizon tasks at 10$\times$ lower retrieval cost. Finally, we instantiate RAVEN on a Unitree Go1 robot for the task of long-horizon navigation for natural language goal-reaching, and show successful deployment over several large indoor environments.

Problem

• long-horizon reasoning
• robotic navigation
• visuo-spatio-temporal memory
• semantic retrieval
• visual embeddings

Innovation

• visuo-spatio-temporal memory
• visual embeddings
• long-horizon navigation
• vector database
• robotic question answering