The Semantic Lifecycle in Embodied AI: Acquisition, Representation and Storage via Foundation Models

2026-01-12

AI Summary

This work addresses the challenge of effectively integrating multi-source, multi-stage semantic information to establish a stable perception-action loop for embodied intelligence in complex, open environments. To overcome the limitations of traditional modular paradigms, the paper proposes a unified “Semantic Lifecycle” framework that conceptualizes semantic knowledge acquisition, representation, and storage as a continuous and dynamic knowledge evolution process. Leveraging the cross-domain generalization capabilities and semantic priors of foundation models, the framework integrates multimodal perception, knowledge representation, and memory mechanisms. The study systematically reviews and contrasts recent advances across the three stages of the semantic lifecycle, identifies key challenges, and provides a cohesive theoretical foundation to advance embodied intelligence toward greater generality and robustness.

Technology Category

• Intelligent Robots: Embodied AI
• Multiagent Systems: Other Foundations of Multi Agent Systems
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Semantic information in embodied AI is inherently multi-source and multi-stage, making it challenging to fully leverage for achieving stable perception-to-action loops in real-world environments. Early studies have combined manual engineering with deep neural networks, achieving notable progress in specific semantic-related embodied tasks. However, as embodied agents encounter increasingly complex environments and open-ended tasks, the demand for more generalizable and robust semantic processing capabilities has become imperative. Recent advances in foundation models (FMs) address this challenge through their cross-domain generalization abilities and rich semantic priors, reshaping the landscape of embodied AI research. In this survey, we propose the Semantic Lifecycle as a unified framework to characterize the evolution of semantic knowledge within embodied AI driven by foundation models. Departing from traditional paradigms that treat semantic processing as isolated modules or disjoint tasks, our framework offers a holistic perspective that captures the continuous flow and maintenance of semantic knowledge. Guided by this embodied semantic lifecycle, we further analyze and compare recent advances across three key stages: acquisition, representation, and storage. Finally, we summarize existing challenges and outline promising directions for future research.

Problem

• Embodied AI
• Semantic Lifecycle
• Foundation Models
• Perception-to-Action Loop
• Semantic Processing

Innovation

• Semantic Lifecycle
• Embodied AI
• Foundation Models
• Semantic Representation
• Knowledge Acquisition