Quotient DAGs for Off-Policy Evaluation:Forward-Flow Importance Sampling and Exact Slate Propensities

2026-05-28

AI Summary

This work addresses the challenge of off-policy evaluation in recommendation settings where the target policy considers unordered item sets while the behavior policy generates ordered interaction trajectories. Standard importance sampling suffers from high variance and computational inefficiency due to redundant permutations and the difficulty of accurately estimating propensities for unordered combinations. To overcome this, the authors model trajectory equivalence classes as a quotient directed acyclic graph (Quotient DAG) and introduce forward flow ratios for weighting. For autoregressive recommenders satisfying set sufficiency, they devise a Forward-DP dynamic programming algorithm that efficiently computes exact propensity scores for unordered sets over a subset DAG. This approach eliminates factorial enumeration, substantially reducing both variance and computational complexity, and presents the first exact and efficient off-policy evaluation method tailored to unordered recommendations.

Abstract

Off-policy evaluation estimates how a target policy would perform using data collected by a different behavior policy, which is crucial when online testing is costly or risky, such as in recommendation or healthcare. Standard importance sampling reweights each logged trajectory, but it can treat details of the generation process as meaningful even when the evaluation target ignores them: for example, an autoregressive slate recommender may generate an ordered sequence of items while the reward and downstream estimator depend only on the unordered slate. This creates nuisance variance and a computational gap, since exact unordered slate propensities require summing over all generation orders. We introduce a quotient-DAG view that merges histories equivalent for evaluation and assigns weights using target-to-behavior forward-flow ratios on the merged graph. For slate recommendation under a set-sufficient next-item interface, this yields Forward-DP, a subset-DAG dynamic program that computes exact unordered propensities without factorial enumeration. The resulting propensity primitive enables practical propensity-based evaluation and model selection for context-dependent autoregressive slate loggers.

Problem

• off-policy evaluation
• importance sampling
• slate recommendation
• propensity score
• nuisance variance

Innovation

• quotient DAG
• off-policy evaluation
• forward-flow importance sampling
• slate recommendation
• exact propensity scoring