Thinking agents for zero-shot generalization to qualitatively novel tasks

2025-03-25

AI Summary

This work addresses how agents can solve qualitatively novel tasks without any real-world environmental interaction—i.e., zero-shot execution—by leveraging mental simulation. Method: We propose a withheld combinatorial task design, wherein test tasks are novel at the compositional level despite comprising previously observed primitive elements, ensuring solvability via internal reasoning. Our core innovation is the first introduction of a “performance delta before vs. after thinking”-driven task selection mechanism, integrated with a world model and mental simulation to enable introspective planning. We further design a thinking-gain–based curriculum learning framework. Contribution/Results: Experiments demonstrate that the agent achieves zero-shot task solving with only a single real-world trial; it significantly outperforms baselines on withheld combinatorial tasks, empirically validating the critical role of mental simulation in generalizing to qualitative novelty.

Technology Category

• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Mechanism Design

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Intelligent organisms can solve truly novel problems which they have never encountered before, either in their lifetime or their evolution. An important component of this capacity is the ability to ``think'', that is, to mentally manipulate objects, concepts and behaviors in order to plan and evaluate possible solutions to novel problems, even without environment interaction. To generate problems that are truly qualitatively novel, while still solvable zero-shot (by mental simulation), we use the combinatorial nature of environments: we train the agent while withholding a specific combination of the environment's elements. The novel test task, based on this combination, is thus guaranteed to be truly novel, while still mentally simulable since the agent has been exposed to each individual element (and their pairwise interactions) during training. We propose a method to train agents endowed with world models to make use their mental simulation abilities, by selecting tasks based on the difference between the agent's pre-thinking and post-thinking performance. When tested on the novel, withheld problem, the resulting agent successfully simulated alternative scenarios and used the resulting information to guide its behavior in the actual environment, solving the novel task in a single real-environment trial (zero-shot).

Problem

• Develop agents for zero-shot novel task generalization
• Train agents to mentally simulate withheld environment combinations
• Enable agents to solve novel tasks without environment interaction

Innovation

• Combinatorial environments for novel tasks
• World models for mental simulation
• Pre-post thinking performance task selection