Bi-Level Contextual Bandits for Individualized Resource Allocation under Delayed Feedback

2025-11-13

AI Summary

Resource allocation in high-stakes domains—such as education, employment, and healthcare—faces challenges including delayed feedback, heterogeneous individual responses, and ethical constraints (e.g., fairness, capacity limits, cooling periods). Method: We propose a two-layer contextual bandit framework: a meta-layer dynamically allocates quotas across subpopulations using subgroup features; a base-layer employs neural networks to identify high-response individuals and models feedback delays via resource-specific delay kernels. The framework integrates contextual bandit learning, neural function approximation, and adaptive budget allocation, enabling policy updates from observational data under dynamic population evolution and operational constraints. Results: Experiments on real-world education and employment datasets demonstrate that our framework significantly improves long-term cumulative reward, more accurately captures delay structures, and achieves robust, fair cross-subgroup allocation under capacity and fairness constraints.

Technology Category

• Machine Learning: Online Learning & Bandits
• Search and Optimization: Learning to Search
• Multiagent Systems: Mechanism Design

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Equitably allocating limited resources in high-stakes domains-such as education, employment, and healthcare-requires balancing short-term utility with long-term impact, while accounting for delayed outcomes, hidden heterogeneity, and ethical constraints. However, most learning-based allocation frameworks either assume immediate feedback or ignore the complex interplay between individual characteristics and intervention dynamics. We propose a novel bi-level contextual bandit framework for individualized resource allocation under delayed feedback, designed to operate in real-world settings with dynamic populations, capacity constraints, and time-sensitive impact. At the meta level, the model optimizes subgroup-level budget allocations to satisfy fairness and operational constraints. At the base level, it identifies the most responsive individuals within each group using a neural network trained on observational data, while respecting cooldown windows and delayed treatment effects modeled via resource-specific delay kernels. By explicitly modeling temporal dynamics and feedback delays, the algorithm continually refines its policy as new data arrive, enabling more responsive and adaptive decision-making. We validate our approach on two real-world datasets from education and workforce development, showing that it achieves higher cumulative outcomes, better adapts to delay structures, and ensures equitable distribution across subgroups. Our results highlight the potential of delay-aware, data-driven decision-making systems to improve institutional policy and social welfare.

Problem

• Allocating limited resources equitably under delayed feedback constraints
• Balancing short-term utility with long-term impact in dynamic populations
• Addressing hidden heterogeneity and ethical constraints in resource allocation

Innovation

• Bi-level contextual bandits for delayed feedback allocation
• Neural network identifies responsive individuals per subgroup
• Delay kernels model treatment effects with cooldown windows