Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study evaluates five small, open-source LLMs for their ability to generate biomedical ontologies by identifying semantic relationships, addressing the bottleneck of manual curation in knowledge management. Using a newly introduced dataset, MeSH-Rel-4K, which contains 4,000 semantic relationships, the authors explore three adaptation strategies: standard prompting, Chain-of-Thought prompting, and fine-tuning. The key finding is that targeted fine-tuning significantly enhances performance, boosting the average F1-score by 34.1 percentage points, thereby demonstrating a viable approach for automating the construction of specialized biomedical ontologies.
Targeted fine-tuning can boost the performance of small LLMs in biomedical ontology generation by over 34 percentage points, revealing a powerful method to automate knowledge structuring.
Knowledge Organization Systems like Ontologies and taxonomies are fundamental for structuring scientific knowledge, yet their manual curation presents a persistent bottleneck in knowledge management. While Large Language Models (LLMs) offer a scalable mechanism for automated ontology generation, their capacity to classify complex, domain-specific semantics requires systematic evaluation. In this paper, we assess the performance of five small, open-source LLMs (up to 9 billion parameters) in identifying semantic relationships between biomedical concepts. To support this evaluation, we introduce MeSH-Rel-4K, a dataset comprising 4K semantic relationships extracted from the Medical Subject Headings (MeSH). We analyse three adaptation strategies: standard prompting, Chain-of-Thought prompting, and fine-tuning. While parameter-constrained models traditionally struggle with the nuances of in-context logic, our results reveal that targeted fine-tuning increases the average F1-score by 34.1 percentage points. These results confirm that direct fine-tuning effectively exceeds the reasoning bottlenecks of smaller LLMs, providing an accurate, automated methodology for the construction and evolution of specialised biomedical ontologies.