Adaptive Cyclic Diffusion for Inference Scaling

2025-05-20

AI Summary

Diffusion models lack the capability to dynamically allocate computational resources per sample based on input difficulty, relying instead on fixed denoising steps. This work introduces an adaptive inference-time scaling paradigm, proposing the Adaptive Bidirectional Cycling Diffusion (ABCD) framework. ABCD features variable-depth bidirectional denoising cycles, a Monte Carlo Tree Search–based cycle search mechanism, an exploration-exploitation balancing strategy, and an online difficulty-aware dynamic termination controller—enabling real-time, sample-specific computational scheduling. Key contributions include: (i) the first formal definition of adaptive computation scheduling at inference time for diffusion models; (ii) the introduction of cyclic search and adaptive “thinking time” mechanisms. Evaluated across multi-task benchmarks, ABCD achieves an average 12.7% performance gain under matched FLOPs and reduces inference steps by up to 38%.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Sampling/Simulation-based Search
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Diffusion models have demonstrated strong generative capabilities across domains ranging from image synthesis to complex reasoning tasks. However, most inference-time scaling methods rely on fixed denoising schedules, limiting their ability to allocate computation based on instance difficulty or task-specific demands adaptively. We introduce the challenge of adaptive inference-time scaling-dynamically adjusting computational effort during inference-and propose Adaptive Bi-directional Cyclic Diffusion (ABCD), a flexible, search-based inference framework. ABCD refines outputs through bi-directional diffusion cycles while adaptively controlling exploration depth and termination. It comprises three components: Cyclic Diffusion Search, Automatic Exploration-Exploitation Balancing, and Adaptive Thinking Time. Experiments show that ABCD improves performance across diverse tasks while maintaining computational efficiency.

Problem

• Adaptive inference-time scaling for diffusion models
• Dynamic computational effort adjustment during inference
• Balancing exploration and exploitation in diffusion cycles

Innovation

• Bi-directional diffusion cycles refine outputs adaptively
• Automatic exploration-exploitation balancing optimizes computation
• Adaptive thinking time controls termination dynamically