Bootstrap Flow-Map Tree Sampling Enables Online Feedback Driven Search

2026-07-02

AI Summary

This work addresses the challenge of efficiently discovering high-value regions under limited sampling budgets and unknown preferences, where existing methods struggle to balance global exploration and local optimization. The authors propose a history-aware global search and alignment framework that introduces a novel Bootstrap Flow-Map-Tree structure, enabling the generation of complete trajectories from any tree depth with a single function evaluation. This design substantially reduces computational overhead while enhancing forward-looking sampling capabilities. By integrating generative models with a tree-based sampling architecture and incorporating a dynamic time-step scheduling mechanism, the method achieves an efficient, history-informed sequential sampling strategy. Experimental results demonstrate significant improvements over current baselines across diverse search and alignment tasks, yielding higher discovery efficiency of valuable regions and stronger adaptability to online feedback.

Abstract

In many scientific and engineering domains, maximizing discovery within a limited sampling budget demands strategic, observation-guided exploration. While generative models have enabled training-free reward alignment, current methods typically excel in local searches within narrow regions of the underlying distribution. These approaches struggle when preferences are unknown a priori and only revealed through sequential feedback-a scenario demanding broad exploration to uncover high-utility regions. To address this, we introduce Bootstrap Flow-Map-Tree (a.k.a BFMT), a novel computationally efficient sampling framework designed for history-aware global search and alignment under sampling budget constraints. BFMT enables full tree-path construction from any tree depth using a single function evaluation, drastically reducing computational overhead while providing critical foresight for sequential sampling. By enabling dynamic transition time steps scheduling, BFMT efficiently allocates its sampling budget, smoothly transitioning from broad global exploration to fine-grained local refinement of high-utility modes discovered through exploration. Extensive experiments and ablations across diverse search and alignment tasks demonstrate that BFMT substantially outperforms baseline approaches.

Problem

• sampling budget
• sequential feedback
• global exploration
• preference uncertainty
• high-utility regions

Innovation

• Bootstrap Flow-Map-Tree
• history-aware sampling
• global exploration
• dynamic scheduling
• sampling efficiency