FIRE3D: Feed-forward Interactive 3D Scene Reconstruction Within A Minute

2026-09-08

AI Summary

FIRE3D框架通过前馈端到端网络，从单张RGB图像或随意拍摄的RGB视频快速生成可用于游戏和交互应用的3D场景资源，解决了实时性和互动性问题。

Abstract

We present FIRE3D, a unified framework that takes a single RGB image or casual RGB video and transforms it into simulation-ready 3D scene assets for games and interactive applications in under a minute. At the core of FIRE3D is a feed-forward, end-to-end network that predicts a compositional scene representation from posed RGB-D observations estimated from the RGB capture, including the 6-DoF pose, bounding box, mesh, and texture for every object. By modeling the scene as a collection of discrete entities, FIRE3D produces amodally complete and simulation-ready environments where objects are physically decoupled and ready for interaction. Our framework requires no test-time optimization, runs orders of magnitude faster than prior interaction-ready methods, and provides object-level completeness beyond existing feed-forward 3D approaches. We demonstrate competitive or state-of-the-art results across pose accuracy, geometry completeness, and texture quality across various datasets while being orders of magnitudes faster. Project page: https://xiahongchi.github.io/Fire3D/

Problem

• 3D Scene Reconstruction
• RGB Video
• Interactive Applications
• Simulation-Ready Assets

Innovation

• feed-forward network
• end-to-end
• compositional scene representation
• simulation-ready 3D assets
• real-time processing