WorldWander: Bridging Egocentric and Exocentric Worlds in Video Generation

2025-11-26

AI Summary

This paper addresses the challenge of seamless viewpoint transition between egocentric (first-person) and exocentric (third-person) perspectives in video generation. To this end, we propose a novel multi-view joint modeling framework. Our method introduces (1) an in-context perspective alignment mechanism to ensure temporal synchronization across viewpoints, and (2) collaborative position encoding to enhance spatial consistency of agents and scenes. We establish the first context-learning framework tailored for multi-view video generation and release EgoExo-8K, a large-scale, multi-view benchmark dataset. Implemented atop a video diffusion Transformer, our approach enables simultaneous egocentric and exocentric video synthesis. Extensive experiments demonstrate significant improvements in cross-view temporal coherence and visual fidelity on both synthetic and real-world scenes, achieving state-of-the-art performance across multiple benchmarks. This work establishes a new paradigm for viewpoint transfer in embodied AI and world modeling research.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Video diffusion models have recently achieved remarkable progress in realism and controllability. However, achieving seamless video translation across different perspectives, such as first-person (egocentric) and third-person (exocentric), remains underexplored. Bridging these perspectives is crucial for filmmaking, embodied AI, and world models. Motivated by this, we present WorldWander, an in-context learning framework tailored for translating between egocentric and exocentric worlds in video generation. Building upon advanced video diffusion transformers, WorldWander integrates (i) In-Context Perspective Alignment and (ii) Collaborative Position Encoding to efficiently model cross-view synchronization. To further support our task, we curate EgoExo-8K, a large-scale dataset containing synchronized egocentric-exocentric triplets from both synthetic and real-world scenarios. Experiments demonstrate that WorldWander achieves superior perspective synchronization, character consistency, and generalization, setting a new benchmark for egocentric-exocentric video translation.

Problem

• Bridging first-person and third-person video perspectives
• Achieving seamless cross-view video translation
• Enhancing synchronization and consistency in multi-perspective videos

Innovation

• In-context learning framework for perspective translation
• Integrates perspective alignment and collaborative position encoding
• Uses large-scale synchronized egocentric-exocentric dataset