Fisher-Preserving Guidance: Training-Free Manifold Constraints for Safe Diffusion Control

2026-05-28

AI Summary

This work addresses the challenge that standard sampling and test-time guidance in diffusion models for visual navigation often produce trajectory updates that deviate from the training data manifold, yielding unreliable or inefficient paths. The authors propose a training-free inference method that combines Fisher information-preserving guidance with subspace projection via outer products to optimize task objectives while suppressing out-of-distribution actions that induce Fisher drift. Introducing, for the first time, Fisher-preserving updates and truncated Fisher denoising sensitivity as uncertainty-aware metrics, the approach enables robust fusion of multi-sample actions. Computational efficiency is further enhanced through low-rank Jacobian decomposition and single-step backpropagation. Evaluated on Maze2D, PushT, and real-world robotic visual navigation benchmarks, the method significantly outperforms existing diffusion-based policies, markedly improving trajectory reliability and task success rates.

Abstract

Diffusion models are effective for waypoint prediction in visual navigation, but standard sampling and test time guidance can produce unreliable or inefficient trajectories when updates drift off the training manifold. We propose Fisher Preserving Guidance with Outer Product Span Projection, a training-free inference method that avoids large Fisher drift associated with off-distribution actions while optimizing a task objective. Our method computes the Fisher-preserving update via a low-rank Jacobian factorization, requiring only a single backward pass per step and enabling real-time use. We further introduce Truncated Fisher Denoising Sensitivity as an uncertainty signal and use it for robust multi-sample action blending. Experiments on toy and realistic navigation benchmarks, including Maze2D with TSDF-based guidance, PushT with official Diffusion Policy weights, and visual navigation in simulation and on real robots, demonstrate consistent improvements in performance over strong diffusion-policy baselines without additional training.

Problem

• diffusion models
• manifold drift
• visual navigation
• trajectory reliability
• test-time guidance

Innovation

• Fisher-Preserving Guidance
• training-free inference
• manifold constraints
• low-rank Jacobian factorization
• uncertainty-aware action blending