Search papers, labs, and topics across Lattice.
This study investigates the use of both general-purpose and medical-adapted LLMs for screening clinical narratives to improve clinical trial recruitment, a known bottleneck. They explored three strategies to mitigate the "Lost in the Middle" issue when handling long documents: original long-context, NER-based extractive summarization, and RAG. Results on the 2018 N2C2 Track 1 benchmark dataset show that MedGemma with RAG achieved the best micro-F1 score of 89.05%, demonstrating improved performance on criteria requiring long-term reasoning.
Medical LLMs using retrieval augmentation (RAG) can now screen clinical trial candidates with high accuracy, but only for criteria requiring reasoning across long documents.
Screening patients for enrollment is a well-known, labor-intensive bottleneck that leads to under-enrollment and, ultimately, trial failures. Recent breakthroughs in large language models (LLMs) offer a promising opportunity to use artificial intelligence to improve screening. This study systematically explored both encoder- and decoder-based generative LLMs for screening clinical narratives to facilitate clinical trial recruitment. We examined both general-purpose LLMs and medical-adapted LLMs and explored three strategies to alleviate the"Lost in the Middle"issue when handling long documents, including 1) Original long-context: using the default context windows of LLMs, 2) NER-based extractive summarization: converting the long document into summarizations using named entity recognition, 3) RAG: dynamic evidence retrieval based on eligibility criteria. The 2018 N2C2 Track 1 benchmark dataset is used for evaluation. Our experimental results show that the MedGemma model with the RAG strategy achieved the best micro-F1 score of 89.05%, outperforming other models. Generative LLMs have remarkably improved trial criteria that require long-term reasoning across long documents, whereas trial criteria that span a short piece of context (e.g., lab tests) show incremental improvements. The real-world adoption of LLMs for trial recruitment must consider specific criteria for selecting among rule-based queries, encoder-based LLMs, and generative LLMs to maximize efficiency within reasonable computing costs.
