XEmbodied: A Foundation Model with Enhanced Geometric and Physical Cues for Large-Scale Embodied Environments

2026-04-20

AI Summary

Current vision-language models lack the capacity to model 3D geometric structures and physical interaction semantics, hindering accurate spatial reasoning required for embodied intelligence. This work addresses this limitation by deeply integrating geometric and physical cues into a cloud-based foundation model. We propose a structured 3D Adapter and a contextual physical token mechanism, combined with an efficient image-to-embodiment adapter, progressive domain curriculum learning, and reinforcement learning-based post-training, to achieve a unified representation of occupancy grids and 3D bounding boxes. Evaluated across 18 public benchmarks, our approach significantly advances performance in spatial reasoning, traffic semantic understanding, embodied affordance recognition, and out-of-distribution generalization, enabling large-scale scene exploration and embodied visual question answering.

Technology Category

• Computer Vision: Large Vision Models
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Intelligent Robots: Embodied AI

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Vision-Language-Action (VLA) models drive next-generation autonomous systems, but training them requires scalable, high-quality annotations from complex environments. Current cloud pipelines rely on generic vision-language models (VLMs) that lack geometric reasoning and domain semantics due to their 2D image-text pretraining. To address this mismatch, we propose XEmbodied, a cloud-side foundation model that endows VLMs with intrinsic 3D geometric awareness and interaction with physical cues (e.g., occupancy grids, 3D boxes). Instead of treating geometry as auxiliary input, XEmbodied integrates geometric representations via a structured 3D Adapter and distills physical signals into context tokens using an Efficient Image-Embodied Adapter. Through progressive domain curriculum and reinforcement learning post-training, XEmbodied preserves general capabilities while demonstrating robust performance across 18 public benchmarks. It significantly improves spatial reasoning, traffic semantics, embodied affordance, and out-of-distribution generalization for large-scale scenario mining and embodied VQA.

Problem

• Vision-Language-Action
• geometric reasoning
• physical cues
• embodied environments
• out-of-distribution generalization

Innovation

• geometric reasoning
• physical cues
• 3D adapter
• embodied VQA
• foundation model