$mathtt{M^3VIR}$: A Large-Scale Multi-Modality Multi-View Synthesized Benchmark Dataset for Image Restoration and Content Creation

2025-09-20

AI Summary

Existing research on game image restoration and controllable generation is hindered by narrow domain focus, artificial degradation artifacts, and the absence of multi-task benchmarks. To address these limitations, we introduce $mathtt{M^3VIR}$, the first large-scale multimodal synthetic dataset tailored for gaming and entertainment applications. Rendered using Unreal Engine 5 across 80 diverse scenes (spanning eight categories), it provides authentic low-resolution/high-resolution image pairs, multi-view video frames, and multi-style renderings. We pioneer object-level controllable video generation and establish a unified evaluation benchmark covering super-resolution, novel view synthesis, and their joint tasks. Two specialized subsets are released: $mathtt{M^3VIR_MR}$ for multi-view restoration and $mathtt{M^3VIR_{MS}}$ for multi-style controllable generation—both significantly enhancing realism and diversity. Extensive experiments demonstrate $mathtt{M^3VIR}$’s effectiveness in rigorously evaluating state-of-the-art methods across all target tasks.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Web data generation and simulation
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The gaming and entertainment industry is rapidly evolving, driven by immersive experiences and the integration of generative AI (GAI) technologies. Training such models effectively requires large-scale datasets that capture the diversity and context of gaming environments. However, existing datasets are often limited to specific domains or rely on artificial degradations, which do not accurately capture the unique characteristics of gaming content. Moreover, benchmarks for controllable video generation remain absent. To address these limitations, we introduce $mathtt{M^3VIR}$, a large-scale, multi-modal, multi-view dataset specifically designed to overcome the shortcomings of current resources. Unlike existing datasets, $mathtt{M^3VIR}$ provides diverse, high-fidelity gaming content rendered with Unreal Engine 5, offering authentic ground-truth LR-HR paired and multi-view frames across 80 scenes in 8 categories. It includes $mathtt{M^3VIR_MR}$ for super-resolution (SR), novel view synthesis (NVS), and combined NVS+SR tasks, and $mathtt{M^3VIR_{MS}}$, the first multi-style, object-level ground-truth set enabling research on controlled video generation. Additionally, we benchmark several state-of-the-art SR and NVS methods to establish performance baselines. While no existing approaches directly handle controlled video generation, $mathtt{M^3VIR}$ provides a benchmark for advancing this area. By releasing the dataset, we aim to facilitate research in AI-powered restoration, compression, and controllable content generation for next-generation cloud gaming and entertainment.

Problem

• Lack of large-scale datasets capturing authentic gaming environment diversity
• Absence of benchmarks for controllable video generation research
• Existing datasets rely on artificial degradations not representing gaming content

Innovation

• Unreal Engine 5 rendered high-fidelity gaming content
• Provides authentic LR-HR paired and multi-view frames
• First multi-style object-level set for controlled video generation