POLARIS: Projection-Orthogonal Least Squares for Robust and Adaptive Inversion in Diffusion Models

2025-11-29

AI Summary

In diffusion model denoising, layer-wise accumulation of approximation errors in noise estimation degrades reconstruction quality. This work first characterizes the propagation mechanism of such errors throughout the reverse process and proposes a novel source-driven error control paradigm—replacing conventional cumulative compensation. We mathematically model the noise approximation bias at each denoising step and introduce a step-wise adaptable guidance scale ω, optimized per layer via projection-based orthogonal least squares. The method improves latent-space reconstruction fidelity with only a one-line code modification, significantly enhancing perceptual fidelity in image editing and inpainting tasks. It achieves state-of-the-art performance across multiple benchmarks without incurring additional computational overhead. The core innovation lies in the synergistic design of error溯源 modeling (i.e., tracing approximation errors to their origin) and lightweight, step-adaptive scale modulation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The Inversion-Denoising Paradigm, which is based on diffusion models, excels in diverse image editing and restoration tasks. We revisit its mechanism and reveal a critical, overlooked factor in reconstruction degradation: the approximate noise error. This error stems from approximating the noise at step t with the prediction at step t-1, resulting in severe error accumulation throughout the inversion process. We introduce Projection-Orthogonal Least Squares for Robust and Adaptive Inversion (POLARIS), which reformulates inversion from an error-compensation problem into an error-origin problem. Rather than optimizing embeddings or latent codes to offset accumulated drift, POLARIS treats the guidance scale ω as a step-wise variable and derives a mathematically grounded formula to minimize inversion error at each step. Remarkably, POLARIS improves inversion latent quality with just one line of code. With negligible performance overhead, it substantially mitigates noise approximation errors and consistently improves the accuracy of downstream tasks.

Problem

• Addresses noise approximation error accumulation in diffusion model inversion
• Reformulates inversion as error-origin problem to minimize step-wise errors
• Improves inversion quality and downstream task accuracy with minimal overhead

Innovation

• Reformulates inversion as error-origin problem
• Treats guidance scale as step-wise variable
• Minimizes inversion error at each step