Playing ZendoWorld: Challenging AI Agents on Active Visual Concept Induction

2026-07-09

AI Summary

This study investigates how agents can actively perceive complex visual inputs, formulate hypotheses, and design informative experiments to infer implicit logical rules. To this end, we introduce ZendoWorld, an interactive environment where agents must deduce hidden rules of a visual game by generating novel scenes and receiving feedback. Our agent integrates vision-language models, Bayesian particle filtering, dynamic concept discovery, and neuro-symbolic reasoning, and its active concept induction capabilities are evaluated through comparison with human behavior. The findings reveal that high predictive accuracy does not necessarily entail faithful rule recovery, that perception and induction constitute critical bottlenecks, and that current large models struggle to design experiments rich in information gain. These results highlight a significant gap in AI’s ability to discover complex rules and offer guidance for developing agents capable of scientific discovery.

Abstract

A central challenge in building intelligent systems is enabling agents to jointly perceive complex inputs, form hypotheses about hidden patterns, and design informative experiments to test them. To study this problem, we propose ZendoWorld, a controlled interactive environment in which agents must infer a logical rule about visual game observations, acquire information by proposing new scenes, and refine their hypotheses based on feedback from the game environment. We evaluate several agents spanning pure VLM reasoning, Bayesian particle filtering, dynamic concept discovery, and neuro-symbolic methods. Our main findings are: (1) high accuracy in predicting labels for observed examples does not imply recovery of the underlying rule; (2) perception and induction are distinct bottlenecks for different agent classes; and (3) VLM-based agents propose near-uninformative experiments, failing to actively reduce hypothesis uncertainty. To compare these results, we collect human data on the task, which reveals a gap in inductive reasoning, particularly for more complex rules. Overall, ZENDOWORLD takes an important step toward evaluating intelligent agents and identifies concrete avenues for improvement, particularly in domains like scientific discovery.

Problem

• active learning
• visual concept induction
• hypothesis testing
• artificial intelligence
• inductive reasoning

Innovation

• active learning
• visual concept induction
• neuro-symbolic reasoning
• Bayesian inference
• interactive environment