Search papers, labs, and topics across Lattice.
This paper introduces BioMedHop, a comprehensive benchmark designed to evaluate biomedical question answering through multi-source evidence and structured reasoning. The benchmark includes 10,045 instances that require complex reasoning tasks, such as shared-neighbor matching and path-based reasoning, across various evidence types. The authors also present BioWeave, a source-aware reasoning framework that significantly enhances performance, achieving a 10.5% improvement over the leading baseline and enabling smaller models to match the reasoning capabilities of larger ones like GPT-4-Turbo.
BioWeave enables smaller models to achieve reasoning performance on par with state-of-the-art LLMs, revolutionizing the landscape of biomedical question answering.
Biomedical question answering (QA) increasingly requires reasoning over interacting entities, where supporting evidence is scattered across biomedical knowledge graphs, literature documents, and web-accessible resources. However, existing biomedical QA benchmarks mainly focus on exam-style knowledge, literature comprehension, or short-range multi-hop inference, leaving source-conditioned graph reasoning and evidence topology construction underexplored. To fill this gap, we introduce BioMedHop, a multi-source graph-grounded benchmark for evaluating biomedical reasoning over structured evidence topologies. BioMedHop contains 10,045 instances across KG, document, web, and hybrid evidence settings, covering shared-neighbor matching, intersection reasoning, path-based reasoning, and counting, with option-based, open-ended, and numeric count renderings. To support this benchmark, we further propose BioWeave, a source-aware reasoning framework that retrieves biomedical KG paths, gathers supporting clues from documents and web sources, assembles them into a unified evidence graph, and verifies answers through entity-level evidence support. Comprehensive experiments show that BioWeave achieves the best overall performance among compared methods on BioMedHop, outperforming the strong hybrid baseline ToG-2 by 10.5% in the overall average. Moreover, BioWeave consistently improves different LLM backbones and enables smaller models, such as Qwen3-4B, to achieve reasoning performance comparable to GPT-4-Turbo.
