Direct Acceleration of Stochastic Root-Finding Without Variance Reduction and Regularization

2026-08-12

AI Summary

This work addresses the challenge that traditional acceleration techniques for stochastic root-finding often fail in stochastic settings due to error accumulation. To overcome this limitation, we propose a novel dual-anchor acceleration mechanism that, for the first time, successfully extends acceleration to the stochastic regime without relying on variance reduction or recursive regularization. The method applies to operators that are cocoercive or square-integrably nonexpansive in expectation and effectively circumvents error accumulation while maintaining a constant batch size. Theoretical analysis shows that the algorithm achieves a sample complexity of $O(\varepsilon^{-3})$ in the general case and improves to nearly optimal $\widetilde{O}(\varepsilon^{-2})$ under strong monotonicity.

Abstract

Acceleration for deterministic root-finding problems has been extensively studied in recent years; specifically, the anchor-based, or Halpern-type methods achieve optimal convergence rates with respect to the operator norm. However, acceleration via these methods does not directly carry over to stochastic setting due to accumulation of errors, unless one enforces diminishing variance via increasing batch sizes or variance reduction techniques. In this work, we show that another class of acceleration, namely the dual-anchor mechanism, extends to the stochastic setting without such error accumulation, in contrast to anchor-based algorithms. Consequently, we cleanly achieve $O(ε^{-3})$ complexity with iteration-independent batch size, without any variance reduction or double-loop recursive regularization, for stochastic root-finding (resp. fixed-point) problems with cocoercivity (resp. square-nonexpansivity) in expectation. For strongly monotone operators, the same algorithm attains a sharper $\widetilde{O} (ε^{-2})$ complexity, nearly matching the lower bound in terms of $ε$-dependence.

Problem

• stochastic root-finding
• acceleration
• variance reduction
• error accumulation
• cocoercivity

Innovation

• dual-anchor mechanism
• stochastic root-finding
• acceleration without variance reduction
• fixed batch size
• cocoercivity