$How^{2}$: How to learn from procedural How-to questions

2025-10-13

AI Summary

AI agents struggle to effectively formulate and leverage “How-to” questions in open-domain planning to mitigate knowledge uncertainty. Method: This paper proposes $How^{2}$, the first question-driven lifelong learning memory agent framework. Its core innovation lies in abstracting and decoupling “How-to” answers from concrete states, enabling cross-task knowledge reuse; it integrates large language models with a hierarchical memory architecture, where multi-level teachers generate subgoals or action sequences within the Plancraft environment. Contribution/Results: Experiments demonstrate that $How^{2}$ significantly improves planning efficiency and generalization for long-horizon tasks, outperforming baselines relying solely on concrete action directives. It achieves a key advance at the intersection of open-domain planning and continual learning.

Technology Category

• Multiagent Systems: Multiagent Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

An agent facing a planning problem can use answers to how-to questions to reduce uncertainty and fill knowledge gaps, helping it solve both current and future tasks. However, their open ended nature, where valid answers to "How do I X?" range from executable actions to high-level descriptions of X's sub-goals, makes them challenging for AI agents to ask, and for AI experts to answer, in ways that support efficient planning. We introduce $How^{2}$, a memory agent framework that enables agents to ask how-to questions, store the answers, and reuse them for lifelong learning in interactive environments. We evaluate our approach in Plancraft, a Minecraft crafting environment, where agents must complete an assembly task by manipulating inventory items. Using teacher models that answer at varying levels of abstraction, from executable action sequences to high-level subgoal descriptions, we show that lifelong learning agents benefit most from answers that are abstracted and decoupled from the current state. $How^{2}$ offers a way for LLM-based agents to improve their planning capabilities over time by asking questions in interactive environments.

Problem

• Agents struggle with procedural how-to questions for planning
• Open-ended answers challenge AI agents and experts
• Lifelong learning agents need abstracted answers for improvement

Innovation

• Memory agent framework for lifelong learning
• Abstracted answers decoupled from current state
• LLM-based agents improve planning in environments