Meta-Object: Interactive and Multisensory Virtual Object Learned from the Real World for the Post-Metaverse

2024-04-26

arXiv.org

AI Summary

To address the lack of realistic physical properties and multimodal interactivity in virtual objects within mixed reality, this paper introduces the “meta-object”—a novel paradigm for next-generation virtual entities capable of inheriting the morphology, physics, and functionality of real-world objects. Methodologically, we propose a post-metaverse platform architecture integrating physics-based modeling, adaptive multisensory feedback generation, and AI-driven simulation—marking the first integration of real-world perceptual characteristics into the meta-object’s core. Our key contribution is enabling spatiotemporally consistent embodied collaborative interaction, validated in AR/VR environments through synchronized haptic-auditory-visual feedback that significantly enhances immersion and cross-modal consistency. Experiments demonstrate a substantial reduction in the experiential gap between virtual and physical objects, establishing a scalable, interoperable infrastructure for intelligent simulation in the post-metaverse era.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

With the proliferation of wearable Augmented Reality/Virtual Reality (AR/VR) devices, ubiquitous virtual experiences seamlessly integrate into daily life through metaverse platforms. To support immersive metaverse experiences akin to reality, we propose a next-generation virtual object, a meta-object, a property-embedded virtual object that contains interactive and multisensory characteristics learned from the real world. Current virtual objects differ significantly from real-world objects due to restricted sensory feedback based on limited physical properties. To leverage meta-objects in the metaverse, three key components are needed: meta-object modeling and property embedding, interaction-adaptive multisensory feedback, and an intelligence simulation-based post-metaverse platform. Utilizing meta-objects that enable both on-site and remote users to interact as if they were engaging with real objects could contribute to the advent of the post-metaverse era through wearable AR/VR devices.

Problem

• Creating virtual objects with real-world form and properties
• Enabling interactive and multisensory AR/VR experiences
• Developing scalable ecosystem for post-metaverse integration

Innovation

• Interactive multisensory virtual objects from real world
• Property-embedded modeling for physical realism
• Scene graph-based intelligence simulation platform