StepGuard: Guarding Web Navigation via Single-Step Calibration

2026-06-16

AI Summary

This work addresses the fragility of single-step decision-making in web navigation agents caused by reward misalignment and error propagation. To this end, the authors propose a dynamic dual-policy optimization framework that enables precise action calibration through a confidence-guided, adaptive reflection mechanism. The approach innovatively alternates between navigation-priority and answer-priority modes to mitigate reward conflicts and incorporates an on-demand, contrastive reward-driven self-correction module. This module synergistically integrates vision-language models, reinforcement learning, and confidence estimation. Evaluated on standard web navigation benchmarks, the method significantly improves both navigation success rate and answer accuracy, achieving state-of-the-art performance.

Abstract

Web navigation requires agents to follow natural language goals, interact with web pages, and produce accurate answers. While recent advances leverage vision-language models and reinforcement learning, existing methods still suffer from single-step fragility due to reward misalignment and error propagation. To tackle the reward entanglement, we design Dynamic Dual-Policy Optimization (DDPO), which dynamically switches between a navigation-first mode for exploration and an answer-first mode for question-answering to mitigate reward conflict. To calibrate the single-step error, we propose Confidence-Guided Adaptive Navigation Reflection (CANR), a mechanism that estimates per-step confidence, triggers reflection only when necessary, and uses contrastive rewards to encourage self-correction to calibrate the single-step inaccuracy. With the above as the main components, we finally develop our StepGuard, a new framework of Guarding Web Navigation via Single-Step Calibration. Experiments demonstrate that our approach significantly improves navigation and answer accuracy, setting new state-of-the-art performance on standard web navigation benchmarks.

Problem

• web navigation
• single-step fragility
• reward misalignment
• error propagation
• vision-language models

Innovation

• Dynamic Dual-Policy Optimization
• Confidence-Guided Adaptive Navigation Reflection
• Single-Step Calibration
• Web Navigation
• Reward Disentanglement