Search papers, labs, and topics across Lattice.
This paper introduces Station, an open-world environment for simulating scientific ecosystems, to investigate AI's capacity for open-ended scientific discovery. By augmenting the environment with Supervisor and Meta Reflection mechanisms, the authors enable persistent exploration in the absence of clear intermediate metrics. Station successfully rediscovers 62.7% of scientific criteria from ICLR papers, significantly outperforming existing multi-agent systems and demonstrating potential for autonomous scientific progress.
AI agents in a simulated scientific ecosystem rediscovered over 60% of key findings from recent research papers, suggesting a viable path to autonomous open-ended scientific discovery.
Recent AI systems have made rapid progress in scientific discovery when given well-defined metrics, but whether they can autonomously undertake open-ended scientific discovery remains unclear. We investigate AI's ability to tackle open-ended tasks in Station, an open-world environment in which multiple agents simulate a scientific ecosystem. To tackle challenges specific to open-ended tasks, we propose augmenting Station with two mechanisms: a Supervisor mechanism and periodic Meta Reflection, which encourage persistent exploration even when intermediate metrics are lacking. We construct open-ended tasks from three recent oral papers presented at ICLR. We give agents the main research question studied in each paper while withholding the paper's results and disabling web access. We then measure how many of the original findings-partitioned into individual criteria-agents rediscover. We find that Station rediscovers 62.7% of the criteria on average, compared with 15.4% for Codex Multiagent-v2 and 14.4-20.6% for AI Scientist-v2. Ablation and behavioral analyses indicate that adding the two mechanisms together improves research coverage and continuity. We further evaluate Station on two open-ended tasks without oracle papers and find that some of the discoveries made by the agents closely match discoveries reported by researchers after the knowledge cutoff date. Together, these results indicate that a suitable environment can enable agents to autonomously make meaningful progress in open-ended scientific discovery.
