Search papers, labs, and topics across Lattice.
This paper adapts a mixture-of-experts diffusion language model, DiffusionGemma-26B, for interactive radiology report drafting and benchmarks it against the autoregressive model Gemma-4-26B. The study finds that the diffusion model not only matches or exceeds the autoregressive counterpart in performance on medical visual question answering tasks but also offers a unique drafting capability through any-order infill, significantly enhancing usability for radiologists. Additionally, the diffusion model demonstrates a decoding speed that is 3.5-4.4 times faster than its autoregressive equivalent, making it a practical choice for real-world applications.
Diffusion models can outperform autoregressive counterparts in medical report drafting while offering a unique any-order infill capability that enhances usability for clinicians.
Diffusion language models, which generate text by denoising a token canvas bidirectionally instead of emitting tokens left to right, have become competitive with autoregressive (AR) generation. Medical foundation models, however, remain almost entirely autoregressive. We adapt a mixture-of-experts diffusion language model, DiffusionGemma-26B, and benchmark it against its same-size AR sibling Gemma-4-26B under an identical LoRA recipe on medical visual question answering datasets, scored by a verbosity-robust LLM judge. Diffusion matches or exceeds AR on all of them, and the finetuned model (3.8B active) is competitive with frontier vision-language models; its decoding is also 3.5-4.4x faster. Beyond this parity, the diffusion model offers a drafting capability AR lacks: any-order infill. Because the canvas is denoised bidirectionally, a radiologist can fix report fragments and have the model fill the text between them, an operation inherent to diffusion but not to autoregression, which is subpar at it. This suits real reports, which are often terse or inconsistent across clinicians and institutions.
Stanford HAI