Search papers, labs, and topics across Lattice.
This paper introduces ZendoWorld, an interactive environment designed to evaluate AI agents' abilities in visual concept induction through hypothesis formation and testing. The study reveals that high accuracy in labeling does not equate to understanding the underlying logical rules, highlighting distinct challenges in perception and induction across different agent types. Notably, VLM-based agents struggle to propose informative experiments, underscoring a critical gap in their ability to reduce hypothesis uncertainty effectively.
VLM-based agents often miss the mark by proposing experiments that fail to clarify their hypotheses, revealing a significant gap in their reasoning capabilities.
A central challenge in building intelligent systems is enabling agents to jointly perceive complex inputs, form hypotheses about hidden patterns, and design informative experiments to test them. To study this problem, we propose ZendoWorld, a controlled interactive environment in which agents must infer a logical rule about visual game observations, acquire information by proposing new scenes, and refine their hypotheses based on feedback from the game environment. We evaluate several agents spanning pure VLM reasoning, Bayesian particle filtering, dynamic concept discovery, and neuro-symbolic methods. Our main findings are: (1) high accuracy in predicting labels for observed examples does not imply recovery of the underlying rule; (2) perception and induction are distinct bottlenecks for different agent classes; and (3) VLM-based agents propose near-uninformative experiments, failing to actively reduce hypothesis uncertainty. To compare these results, we collect human data on the task, which reveals a gap in inductive reasoning, particularly for more complex rules. Overall, ZENDOWORLD takes an important step toward evaluating intelligent agents and identifies concrete avenues for improvement, particularly in domains like scientific discovery.
