DAGR: State-Conditioned Goal Representations via Difference-Aware Goal Cross-Attention

2026-07-15

AI Summary

This work addresses a key limitation in existing goal-conditioned reinforcement learning methods, which often neglect the current state and thereby struggle to discern which parts of the goal remain to be executed. To overcome this, the authors propose a multi-scale gated cross-attention mechanism that dynamically transforms static goal embeddings into state-conditioned representations. This is achieved through difference-aware attention biasing and near-identity gated residual connections, which explicitly model the discrepancy between state and goal while preserving original information. The approach seamlessly integrates with any late-fusion goal encoder and yields significant performance gains on OGBench navigation tasks—primarily attributable to the gated residual pathway—while achieving comparable or slightly inferior results on manipulation and puzzle-solving tasks, thereby validating its effectiveness as a targeted improvement for navigation-oriented scenarios.

Abstract

Goal-conditioned reinforcement learning hinges on how the goal is encoded. Contrastive, metric, temporal-distance, and information-theoretic encoders differ in objective. They still share one trait. None of them sees the current state. Such a state-independent embedding cannot mark which part of the goal still needs action. The policy must then recover that cue by inverting both encoders. We propose DAGR. It refines the static embedding of any late-fusion encoder into a state-conditioned one through multi-scale gated cross-attention. A near-identity gated residual preserves the base representation. Difference-aware Goal Cross-Attention then biases the attention scores using a per-token state-goal discrepancy map. On OGBench, DAGR improves navigation. Our ablations trace the gain to the gated residual, not to the difference bias that names the method. On manipulation and puzzle tasks it matches or falls below the base. DAGR is a structured refinement, not a universal improvement.

Problem

• goal-conditioned reinforcement learning
• state-independent embedding
• goal representation
• action relevance
• state-goal discrepancy

Innovation

• state-conditioned goal representation
• difference-aware cross-attention
• goal-conditioned reinforcement learning
• gated residual
• multi-scale attention