NymeriaPlus: Enriching Nymeria Dataset with Additional Annotations and Data

2026-03-19

AI Summary

Existing first-person perspective datasets are limited in multimodal annotation, 3D object reconstruction, and human motion representation, hindering progress in embodied intelligence research. To address these shortcomings, this work introduces NymeriaPlus, an enhanced dataset built upon Nymeria that, for the first time, integrates high-fidelity human motion (leveraging Momentum Human Rig and SMPL models), dense 2D/3D bounding boxes, instance-level 3D object reconstructions, and additional heterogeneous modalities such as audio and wrist-mounted video within a single in-the-wild egocentric setting. By employing semi-dense point cloud processing, multi-device spatiotemporal synchronization, and cross-modal alignment techniques, NymeriaPlus substantially improves data density and structural coherence, establishing a high-quality, multidimensional benchmark for embodied AI and multimodal learning.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

The Nymeria Dataset, released in 2024, is a large-scale collection of in-the-wild human activities captured with multiple egocentric wearable devices that are spatially localized and temporally synchronized. It provides body-motion ground truth recorded with a motion-capture suit, device trajectories, semi-dense 3D point clouds, and in-context narrations. In this paper, we upgrade Nymeria and introduce NymeriaPlus. NymeriaPlus features: (1) improved human motion in Momentum Human Rig (MHR) and SMPL formats; (2) dense 3D and 2D bounding box annotations for indoor objects and structural elements; (3) instance-level 3D object reconstructions; and (4) additional modalities e.g., basemap recordings, audio, and wristband videos. By consolidating these complementary modalities and annotations into a single, coherent benchmark, NymeriaPlus strengthens Nymeria into a more powerful in-the-wild egocentric dataset. We expect NymeriaPlus to bridge a key gap in existing egocentric resources and to support a broader range of research, including unique explorations of multimodal learning for embodied AI.

Problem

• egocentric vision
• multimodal learning
• 3D object annotation
• embodied AI
• human activity dataset

Innovation

• egocentric vision
• multimodal learning
• 3D object reconstruction
• motion capture
• embodied AI