Memory-Guided View Refinement for Dynamic Human-in-the-loop EQA

2026-03-10

AI Summary

This work addresses the challenge of embodied question answering (EQA) in dynamic human-inhabited environments, where human activity and occlusions induce non-stationary perception, hindering traditional EQA methods from efficiently accumulating reliable, compact, and timely visual evidence. The study presents the first systematic investigation of EQA under such conditions, introducing the DynHiL-EQA dataset and proposing DIVRR—a training-free framework. DIVRR leverages correlation-guided view optimization, selective memory admission, and evidence filtering to validate ambiguous observations without requiring training, retaining only high-information evidence for efficient and robust reasoning. Experiments demonstrate that DIVRR significantly outperforms existing approaches on both DynHiL-EQA and HM-EQA benchmarks, achieving consistently high accuracy and reasoning efficiency across dynamic and static scenarios.

Technology Category

• Knowledge Representation and Reasoning: Qualitative Reasoning
• Intelligent Robots: Embodied AI
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Embodied Question Answering (EQA) has traditionally been evaluated in temporally stable environments where visual evidence can be accumulated reliably. However, in dynamic, human-populated scenes, human activities and occlusions introduce significant perceptual non-stationarity: task-relevant cues are transient and view-dependent, while a store-then-retrieve strategy over-accumulates redundant evidence and increases inference cost. This setting exposes two practical challenges for EQA agents: resolving ambiguity caused by viewpoint-dependent occlusions, and maintaining compact yet up-to-date evidence for efficient inference. To enable systematic study of this setting, we introduce DynHiL-EQA, a human-in-the-loop EQA dataset with two subsets: a Dynamic subset featuring human activities and temporal changes, and a Static subset with temporally stable observations. To address the above challenges, we present DIVRR (Dynamic-Informed View Refinement and Relevance-guided Adaptive Memory Selection), a training-free framework that couples relevance-guided view refinement with selective memory admission. By verifying ambiguous observations before committing them and retaining only informative evidence, DIVRR improves robustness under occlusions while preserving fast inference with compact memory. Extensive experiments on DynHiL-EQA and the established HM-EQA dataset demonstrate that DIVRR consistently improves over existing baselines in both dynamic and static settings while maintaining high inference efficiency.

Problem

• Embodied Question Answering
• dynamic environments
• occlusions
• evidence accumulation
• inference efficiency

Innovation

• Embodied Question Answering
• Dynamic Environments
• View Refinement
• Memory Selection
• Occlusion Handling