Search papers, labs, and topics across Lattice.
This paper evaluates seven open-source LLMs on three tasks relevant to Japanese pathology report writing: structured report generation/extraction, typo correction, and explanatory text generation. Thinking and medical-specialized models excelled at structured reporting and typo correction, while subjective evaluations of explanatory text varied greatly. The study concludes that open-source LLMs can be useful in specific, clinically relevant scenarios for assisting Japanese pathology report writing.
Open-source LLMs can help write Japanese pathology reports, but pathologists strongly disagree on which model provides the best explanations.
The performance of large language models (LLMs) for supporting pathology report writing in Japanese remains unexplored. We evaluated seven open-source LLMs from three perspectives: (A) generation and information extraction of pathology diagnosis text following predefined formats, (B) correction of typographical errors in Japanese pathology reports, and (C) subjective evaluation of model-generated explanatory text by pathologists and clinicians. Thinking models and medical-specialized models showed advantages in structured reporting tasks that required reasoning and in typo correction. In contrast, preferences for explanatory outputs varied substantially across raters. Although the utility of LLMs differed by task, our findings suggest that open-source LLMs can be useful for assisting Japanese pathology report writing in limited but clinically relevant scenarios.
