Search papers, labs, and topics across Lattice.
This paper introduces Autodata, a novel method that empowers AI agents to function as data scientists, generating high-quality training and evaluation datasets. By employing a meta-optimization approach, the authors demonstrate that the agent can create superior synthetic data, leading to significant performance improvements in various tasks, including computer science research and legal reasoning. The results indicate that optimizing the data scientist agent itself yields even greater enhancements, suggesting a transformative potential for data generation in AI development.
Meta-optimizing an AI data scientist can dramatically enhance the quality of synthetic datasets, outperforming traditional methods.
We introduce Autodata, a general method that enables AI agents to act as data scientists who build high quality training and evaluation data. We show how to train (meta-optimize) such a data scientist agent, so that it learns to create even stronger data. We describe the overall formulation, and a specific practical implementation, Agentic Self-Instruct. We conduct experiments on computer science research tasks, legal reasoning tasks and reasoning with mathematical objects, where we obtain improved results compared to classical synthetic dataset creation methods. Further, meta-optimizing the data scientist agent itself delivers an even larger performance uplift. Agentic data creation provides a way to convert increased inference compute into higher quality model training. Overall, we believe this direction has the potential to change the way we build AI data.
Meta AI