Preemptive Solving of Future Problems: Multitask Preplay in Humans and Machines

2025-07-07

AI Summary

Humans face a fundamental tension between vast task spaces and limited concurrent execution capacity, necessitating rapid cross-task generalization. To address this, we propose “multi-task rehearsal”—a mechanism that leverages counterfactual simulation of unexecuted yet semantically related tasks to construct prospective predictive representations, enabling foresight-driven learning grounded in task co-occurrence structure. Our approach integrates reinforcement learning, experience replay, counterfactual reasoning, and predictive coding, and is evaluated in grid-world and partially observable Craftax environments. Compared to conventional planning methods, it achieves significantly improved transfer performance on novel tasks and environments, while better fitting human behavioral data. This work formally defines the multi-task rehearsal algorithm for the first time, revealing a new cross-task generalization mechanism that balances cognitive plausibility with computational scalability.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Humans can pursue a near-infinite variety of tasks, but typically can only pursue a small number at the same time. We hypothesize that humans leverage experience on one task to preemptively learn solutions to other tasks that were accessible but not pursued. We formalize this idea as Multitask Preplay, a novel algorithm that replays experience on one task as the starting point for "preplay" -- counterfactual simulation of an accessible but unpursued task. Preplay is used to learn a predictive representation that can support fast, adaptive task performance later on. We first show that, compared to traditional planning and predictive representation methods, multitask preplay better predicts how humans generalize to tasks that were accessible but not pursued in a small grid-world, even when people didn't know they would need to generalize to these tasks. We then show these predictions generalize to Craftax, a partially observable 2D Minecraft environment. Finally, we show that Multitask Preplay enables artificial agents to learn behaviors that transfer to novel Craftax worlds sharing task co-occurrence structure. These findings demonstrate that Multitask Preplay is a scalable theory of how humans counterfactually learn and generalize across multiple tasks; endowing artificial agents with the same capacity can significantly improve their performance in challenging multitask environments.

Problem

• How humans generalize to unpursued tasks using experience
• Developing Multitask Preplay for adaptive task performance
• Improving AI agent transfer learning in multitask environments

Innovation

• Multitask Preplay algorithm replays task experience
• Preplay simulates unpursued tasks counterfactually
• Predictive representation enables fast adaptive performance