From Extraction to Navigation: Progressive Retrieval with Indirectly Infinite Depth

2026-06-29

AI Summary

Traditional recommendation retrieval is constrained by static graph structures and fixed entry points, often leading to interest tunneling or search drift and failing to capture users’ deeper intent. This work reframes retrieval as a stateful, autonomous graph exploration process and introduces a recursive state reuse mechanism enabling indirect infinite depth (IID). By integrating goal-aware navigation, active intent routing, graph-based hard negative sampling, and a trajectory-aligned training paradigm, the proposed approach overcomes the inherent trade-off between retrieval depth and latency. Evaluated on billion-scale industrial datasets under strict latency constraints, the method significantly outperforms mainstream baselines, effectively mitigating search drift while maintaining high-precision retrieval along deep exploration paths.

Abstract

Modern large-scale recommender retrieval is shifting from static similarity matching to dynamic item space navigation, framing retrieval as iterative goal-driven graph traversal. Conventional item-to-item (i2i) methods fall into the "interest tunnel" and fail to excavate deep user interests, while existing index-based retrieval suffers from persistent "search drift", caused by static entry nodes and fixed graph topologies unable to track shifting real-time user intent. To resolve the above defects, we present IID-Nav, a framework modeling retrieval as stateful autonomous graph exploration with three core contributions: (1) A goal-aware navigation policy substituting passive neighborhood expansion with active intent routing supervised by a target discriminator; (2) A recursive state evolution mechanism supporting Indirectly Infinite Depth (IID) via cross-request state reuse, which enables logical unlimited-depth graph traversal without linearly rising inference latency; (3) A trajectory-aligned training paradigm equipped with graph hard negative sampling to stabilize optimization over full navigation paths. Evaluations on billion-level industrial datasets show IID-Nav surpasses mainstream retrieval baselines under strict latency budgets. Empirical results verify that our method alleviates search drift remarkably and retains high precision for deep retrieval paths, offering an efficient, robust retrieval solution for industrial recommendation systems.

Problem

• interest tunnel
• search drift
• dynamic item space navigation
• real-time user intent
• deep retrieval

Innovation

• graph navigation
• indirectly infinite depth
• stateful retrieval
• goal-aware policy
• trajectory-aligned training