RenderFormer++: Scalable and Physically Grounded Feed-Forward Neural Rendering

2026-06-29

AI Summary

This work addresses the scalability limitations and physical inconsistencies of existing Transformer-based neural rendering methods when generalizing across scenes, which stem from quadratic attention complexity and inadequate adherence to physical principles. To overcome these challenges, we propose a scalable and physically plausible feed-forward neural rendering framework that integrates Physics-Informed Transport Guidance (PITG) attention—incorporating priors from the rendering equation—to enforce global illumination consistency. Coupled with Hierarchical Object-Centric Tokenization (HOCT), our approach enables efficient, multi-scale token representation. The resulting method significantly reduces computational overhead while preserving geometric and radiometric fidelity, thereby enhancing physical consistency, cross-scene generalization, and rendering efficiency. Experiments demonstrate superior performance over state-of-the-art approaches in large-scale, complex scenes.

Abstract

We present RenderFormer++, a scalable and physically grounded feed-forward neural rendering framework for global illumination in mesh scenes. Existing Transformer-based neural rendering methods such as RenderFormer achieve promising cross-scene generalization, but suffer from limited physical consistency and poor scalability due to the quadratic attention complexity of triangle-level tokenization. To address these issues, we introduce Physics-Informed Transport Guidance (PITG), which embeds rendering-equation inductive biases into the attention mechanism and enforces transport consistency loss, enabling physically consistent light transport modeling. We further propose Hierarchical Object-Centric Tokenization (HOCT), which aggregates triangle-level features into compact object-level tokens via cross-attention with learnable queries, substantially reducing computational and memory costs while preserving geometric and radiometric information. Extensive experiments demonstrate that RenderFormer++ achieves scalable, stable, and generalizable feed-forward global illumination rendering across complex large-scale scenes with improved physical accuracy and efficiency over prior neural rendering methods.

Problem

• neural rendering
• global illumination
• physical consistency
• scalability
• Transformer

Innovation

• Physics-Informed Transport Guidance
• Hierarchical Object-Centric Tokenization
• Neural Rendering
• Global Illumination
• Scalable Transformer