KaoLRM: Repurposing Pre-trained Large Reconstruction Models for Parametric 3D Face Reconstruction

2026-01-19

AI Summary

This work addresses the challenge of geometric and appearance inconsistencies in cross-view facial reconstruction caused by self-occlusion and viewpoint variation, which commonly plague existing 3D Morphable Model (3DMM) regressors. The authors propose a novel approach that, for the first time, effectively transfers the triplane-based 3D priors from a pre-trained Large Reconstruction Model (LRM) into the parametric FLAME space. By tightly coupling FLAME mesh geometry with 2D Gaussian primitives, the method jointly models geometry and appearance in a unified framework. This integration significantly enhances cross-view consistency and robustness to self-occlusion in single-view reconstruction. Extensive evaluations on both controlled and real-world datasets demonstrate superior performance over state-of-the-art methods, effectively mitigating view-dependent artifacts and reducing viewpoint sensitivity.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

We propose KaoLRM to re-target the learned prior of the Large Reconstruction Model (LRM) for parametric 3D face reconstruction from single-view images. Parametric 3D Morphable Models (3DMMs) have been widely used for facial reconstruction due to their compact and interpretable parameterization, yet existing 3DMM regressors often exhibit poor consistency across varying viewpoints. To address this, we harness the pre-trained 3D prior of LRM and incorporate FLAME-based 2D Gaussian Splatting into LRM's rendering pipeline. Specifically, KaoLRM projects LRM's pre-trained triplane features into the FLAME parameter space to recover geometry, and models appearance via 2D Gaussian primitives that are tightly coupled to the FLAME mesh. The rich prior enables the FLAME regressor to be aware of the 3D structure, leading to accurate and robust reconstructions under self-occlusions and diverse viewpoints. Experiments on both controlled and in-the-wild benchmarks demonstrate that KaoLRM achieves superior reconstruction accuracy and cross-view consistency, while existing methods remain sensitive to viewpoint variations. The code is released at https://github.com/CyberAgentAILab/KaoLRM.

Problem

• 3D face reconstruction
• 3D Morphable Models
• viewpoint consistency
• single-view reconstruction
• FLAME

Innovation

• Large Reconstruction Model
• Parametric 3D Face Reconstruction
• FLAME
• 2D Gaussian Splatting
• Cross-view Consistency