🤖 AI Summary
This paper addresses “unconscious rearrangement”—a novel interactive task-learning scenario wherein an agent operates in rigid-body environments without prior knowledge of critical conceptual abstractions. We propose a semantic-aware embodied dialogue framework that integrates embodied active questioning, user-correction feedback modeling, logic-driven evidence-augmented reasoning, and corrective reinforcement learning to enable concept discovery, dynamic model refinement, and cross-task knowledge transfer. Our key contribution is the first integration of formal semantic-logical analysis into an embodied dialogue loop, supporting strategic, inference-guided questioning and lifelong learning. Experiments demonstrate significant improvements in data efficiency and strong generalization to previously unseen concepts, empirically validating the pivotal role of semantic enhancement in improving reasoning robustness.
📝 Abstract
This paper addresses a challenging interactive task learning scenario we call rearrangement under unawareness: to manipulate a rigid-body environment in a context where the agent is unaware of a concept that is key to solving the instructed task. We propose SECURE, an interactive task learning framework designed to solve such problems. It uses embodied conversation to fix its deficient domain model -- through dialogue, the agent discovers and then learns to exploit unforeseen possibilities. In particular, SECURE learns from the user's embodied corrective feedback when it makes a mistake, and it makes strategic dialogue decisions to reveal useful evidence about novel concepts for solving the instructed task. Together, these abilities allow the agent to generalise to subsequent tasks using newly acquired knowledge. We demonstrate that learning to solve rearrangement under unawareness is more data efficient when the agent is semantics-aware -- that is, during both learning and inference it augments the evidence from the user's embodied conversation with its logical consequences, stemming from semantic analysis.