Error as Signal: Stiffness-Aware Diffusion Sampling via Embedded Runge-Kutta Guidance

2026-03-03

AI Summary

This work addresses the degradation in sampling quality of diffusion models within stiff regions, which arises from local truncation error (LTE) and is exacerbated by existing guidance methods that fail to effectively exploit solver error information. The authors propose a novel stiffness-aware guidance mechanism that, for the first time, aligns LTE with the dominant eigenvector of the Jacobian as a guidance signal. Leveraging an embedded Runge–Kutta method, the approach dynamically detects stiffness and estimates LTE during integration, enabling adaptive adjustment of the diffusion ODE solver’s sampling trajectory. Notably, this method requires no auxiliary networks and significantly enhances both conditional generation quality and sampling stability. Empirical evaluations on synthetic datasets and ImageNet demonstrate consistent superiority over current state-of-the-art techniques.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Classifier-Free Guidance (CFG) has established the foundation for guidance mechanisms in diffusion models, showing that well-designed guidance proxies significantly improve conditional generation and sample quality. Autoguidance (AG) has extended this idea, but it relies on an auxiliary network and leaves solver-induced errors unaddressed. In stiff regions, the ODE trajectory changes sharply, where local truncation error (LTE) becomes a critical factor that deteriorates sample quality. Our key observation is that these errors align with the dominant eigenvector, motivating us to leverage the solver-induced error as a guidance signal. We propose Embedded Runge-Kutta Guidance (ERK-Guid), which exploits detected stiffness to reduce LTE and stabilize sampling. We theoretically and empirically analyze stiffness and eigenvector estimators with solver errors to motivate the design of ERK-Guid. Our experiments on both synthetic datasets and the popular benchmark dataset, ImageNet, demonstrate that ERK-Guid consistently outperforms state-of-the-art methods. Code is available at https://github.com/mlvlab/ERK-Guid.

Problem

• stiffness
• local truncation error
• diffusion sampling
• ODE solver
• sample quality

Innovation

• Embedded Runge-Kutta Guidance
• stiffness-aware sampling
• local truncation error
• diffusion models
• ODE solvers