Search papers, labs, and topics across Lattice.
This paper introduces ARLtR, a unified framework for automated knowledge graph construction and fact-grounded question generation, addressing the limitations of existing datasets that separate vector-based retrieval and reasoning over knowledge graphs. By integrating knowledge graphs with embeddings and question-answer pairs grounded in extracted entities and relations, ARLtR creates a comprehensive historical dataset focused on the Roman Empire, featuring over 19,000 entities and 8,400 question-answer pairs. The framework enables the development of hybrid retrieval systems and semantic steering approaches, enhancing the evaluation of AI models in information retrieval and question answering.
A unified framework that combines knowledge graphs and question generation could revolutionize how AI systems retrieve and reason about information.
Large language models have substantially improved information retrieval and question answering; however, existing datasets generally support either vector-based retrieval over unstructured text or reasoning over knowledge graphs, without providing a unified representation that combines both paradigms. Moreover, current benchmarks rarely provide ground-truth entities, relations, and fact-grounded question-answer pairs aligned with the underlying corpus. To address this gap, we introduce All Relations Lead to Rome (ARLtR), a unified framework for automated knowledge graph construction and fact-grounded question-answer generation. ARLtR jointly constructs a knowledge graph, embeddings, and question-answer pairs that are explicitly grounded in extracted entities, relations, and supporting textual evidence. We further instantiate the framework as a historical dataset centered on the Roman Empire, comprising over 19,000 entities, 16,000 chunks, and 8,400 question-answer pairs (https://huggingface.co/datasets/FaynePro/all-relations-lead-to-rome). By tightly coupling symbolic graph representations with dense retrieval representations, ARLtR facilitates the evaluation and development of hybrid retrieval systems and semantic steering approaches within a single coherent resource.
ETH