Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study benchmarks six large language models (LLMs) for few-shot classification of inflammatory bowel disease (IBD) from gut microbiome profiles, addressing the challenges posed by zero-inflation and skewed distributions in microbiome data. By evaluating three proprietary and three open-source models under controlled conditions, the research reveals that the open-source Mistral 7B outperforms all proprietary models in Macro-F1 score without requiring parameter updates. The findings highlight that the performance gap between open-source and proprietary models is minimal, suggesting that open-source LLMs are viable alternatives for privacy-sensitive clinical applications.
Open-source LLMs can outperform proprietary counterparts in microbiome IBD classification, challenging the notion that commercial APIs are the only viable option for clinical deployments.
Phenotyping inflammatory bowel disease (IBD) from gut microbiome profiles remains challenging due to 93% genus-level zero-inflation, skewed amplicon count distributions, and the practical cost of assembling labelled cohorts. Few-shot in-context learning (ICL) with large language models (LLMs) sidesteps the annotation bottleneck, yet existing benchmarks test only proprietary APIs—a deployment model incompatible with the data governance constraints of most clinical sites. We benchmark six frontier LLMs under identical few-shot conditions—three proprietary (GPT-4o, Claude claude-opus-4-5, Gemini 2.5 Flash Lite) and three open-source (Mistral 7B, LLaMA-3 8B, DeepSeek-R1-Distill-Qwen 1.5B)—evaluated against supervised baselines (Random Forest, XGBoost, LightGBM, soft-voting Ensemble) on 16S rRNA amplicon data (n = 1,316; holdout n = 30). All six LLMs received identical prompts, the same log-normalised top-20 features, and the same random shot selection strategy with a fixed seed, ensuring that observed differences are attributable to model capacity rather than experimental conditions. The supervised Ensemble led holdout performance (Macro-F1: 0.7948; AUC: 0.7725). Among LLMs, Mistral 7B achieved the highest Macro-F1 (0.5417), surpassing all three proprietary models without any parameter update. The mean performance gap between open-source (0.4711) and proprietary (0.5101) groups was only 0.039 Macro-F1 points—too narrow to justify exclusive reliance on commercial APIs in privacy-sensitive deployments. Within-open-source variance (0.130 Macro-F1 points) substantially exceeded this inter-family gap, indicating that model selection within the open-source ecosystem is the more consequential practical decision. These results suggest that open-source LLMs running on local hardware are a workable option when labeled data is limited and routing patient-derived data to external APIs is not permitted.