Fast and accurate neural reflectance transformation imaging through knowledge distillation

2025-10-28

AI Summary

To address the high computational cost of NeuralRTI, which hinders real-time, full-resolution interactive relighting on resource-constrained hardware, this paper proposes DisK-NeuralRTI—the first knowledge distillation-based lightweight neural reflectance transform imaging framework. Our method integrates a neural autoencoder with polynomial texture mapping and employs spherical harmonics to represent the reflectance field, systematically transferring the high-fidelity modeling capability of a large teacher network to a compact student network. Compared to the original NeuralRTI, DisK-NeuralRTI achieves comparable rendering quality (PSNR > 38 dB) at similar parameter counts, while accelerating inference by 3.2–5.7×. It is the first method to enable real-time (>30 FPS) interactive relighting of 1024×1024 images on consumer-grade GPUs. The core contribution lies in the systematic integration of knowledge distillation into RTI modeling, effectively bridging the performance gap between expressive reflectance field representation and efficient deployment.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Learning with Manifolds
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Reflectance Transformation Imaging (RTI) is very popular for its ability to visually analyze surfaces by enhancing surface details through interactive relighting, starting from only a few tens of photographs taken with a fixed camera and variable illumination. Traditional methods like Polynomial Texture Maps (PTM) and Hemispherical Harmonics (HSH) are compact and fast, but struggle to accurately capture complex reflectance fields using few per-pixel coefficients and fixed bases, leading to artifacts, especially in highly reflective or shadowed areas. The NeuralRTI approach, which exploits a neural autoencoder to learn a compact function that better approximates the local reflectance as a function of light directions, has been shown to produce superior quality at comparable storage cost. However, as it performs interactive relighting with custom decoder networks with many parameters, the rendering step is computationally expensive and not feasible at full resolution for large images on limited hardware. Earlier attempts to reduce costs by directly training smaller networks have failed to produce valid results. For this reason, we propose to reduce its computational cost through a novel solution based on Knowledge Distillation (DisK-NeuralRTI). ...

Problem

• Reducing computational cost of neural reflectance transformation imaging
• Overcoming artifacts in traditional reflectance field modeling
• Enabling high-quality rendering on limited hardware resources

Innovation

• Knowledge distillation reduces NeuralRTI computational cost
• Neural autoencoder learns compact reflectance function
• Improved rendering efficiency maintains superior quality output