🤖 AI Summary
This work addresses the limitations of existing knowledge distillation methods in partially observable environments, where fixed distillation coefficients fail to account for agent uncertainty. The authors propose Belief-Aware Guided Soft Actor-Critic (BA-GSAC), which dynamically adjusts the distillation coefficient based on ensemble model disagreement and systematically evaluates multiple scheduling strategies under varying degrees of observation occlusion. Experiments demonstrate that adaptive guidance significantly improves training stability in mildly to moderately partial observability (coefficient of variation: 13.3% vs. 29.8%), though it tends to degrade to its minimum value under severe occlusion. In such high-occlusion regimes, a linear decay strategy achieves superior performance (mean return: 116.5, coefficient of variation: 8.9%). The study further uncovers an “observability blind spot” in ensemble-based approaches under heavy occlusion and suggests architectural improvements targeting full-state information as a promising direction.
📝 Abstract
Guided Soft Actor-Critic (GSAC) distills knowledge from a privileged full-state teacher to a partial-observation student for autonomous driving, but uses a fixed distillation coefficient lambda regardless of the agent's uncertainty. We present Belief-Aware GSAC (BA-GSAC), which modulates lambda via ensemble disagreement, and use it as a testbed for a systematic empirical study asking: when does adaptive guidance actually help? Evaluating five strategies (fixed lambda in {0.01, 0.1}, adaptive, linear decay, and vanilla SAC) across three POMDP difficulty levels on Highway-Env, we find that preliminary single-seed runs suggest benefits under mild and moderate partial observability, but under severe occlusion (evaluated with 3 seeds for all methods) the adaptive coefficient collapses to lambda_min within about 3K steps. We trace this to an observability blindness phenomenon: because the ensemble predicts partial observations, it achieves low disagreement even under heavy occlusion, modeling what is visible but unable to detect what is missing. We diagnose the root cause and propose an architectural fix (training the ensemble on full-state predictions using the guiding actor's privileged access); while not validated here, we show that even with current limitations, the warmup phase provides measurable stabilization (CV=13.3% vs. 29.8% for constant lambda=0.01). In fact, a simple deterministic linear decay schedule achieves the best severe-POMDP performance across all metrics (mean 116.5, CV=8.9%), suggesting that the scheduling effect, not the ensemble, drives the stability benefit. These findings provide practical guidance for designing uncertainty-aware teacher-student frameworks and highlight ensemble prediction targets as an important design choice.