Search papers, labs, and topics across Lattice.
This paper introduces Modelpedia, an automated framework that leverages large language models to systematically extract and catalog findings related to AI models from a multitude of published sources. By applying this prototype to accepted ICLR 2024 and 2025 papers, the authors successfully aggregated over a thousand findings into a searchable public catalog, enabling easier access to critical insights about model behavior and performance. The meta-analysis conducted on the catalog itself reveals patterns in how the AI community investigates models, highlighting the need for a centralized knowledge repository in a rapidly evolving field.
A searchable catalog of over a thousand AI model findings could revolutionize how researchers access and build upon existing knowledge in the field.
Scientific knowledge about AI models is produced faster than the community can organize it. Every few months a new foundation model reshapes the field and hundreds of papers, blogs, and technical reports document how each behaves or fails. Yet, these findings remain scattered and effectively unretrievable. To address this gap we present Modelpedia, an automated, LLM-assisted framework that extracts findings about models from published papers, links it to the model, dataset, method, and concept it concerns, and aggregates the result into a searchable public catalog. Applying the prototype to accepted ICLR 2024 and 2025 papers, we extract over a thousand findings and, treating the catalog itself as an object of study, run a meta-analysis of how the community investigates models. Now, we invite the community to explore, contribute to, and build on the open catalog, and to help establish model findings as a shared foundation for the meta-science of AI.
