Search papers, labs, and topics across Lattice.
This paper introduces Federated Self-Contextualization (FSC), a novel framework designed for clinical audio diagnosis in low-resource settings, which leverages unsupervised clustering of audio representations to create pseudo-label episodes. By employing a progressive three-stage pipeline, FSC aligns audio embeddings with a language model and adapts it for episodic in-context inference through federated optimization. The approach demonstrates significant efficacy, achieving 71.6% accuracy in diagnosing respiratory and cardiac conditions with minimal labeled examples, surpassing existing audio-language models by over 9%.
Achieving 71.6% accuracy in diagnosing unseen medical conditions with just two labeled examples showcases the power of federated learning in low-resource clinical settings.
Clinical audio diagnosis in low-resource settings requires models that identify conditions from minimal examples without large annotated corpora. We propose Federated Self-Contextualization (FSC), a multimodal language model framework for in-context clinical audio diagnosis across federated hospital clients. FSC constructs pseudo-label episodes via unsupervised clustering of audio representations, bypassing scarce real diagnostic labels, and enables contextual reasoning from support-query pairs. Our progressive three-stage pipeline first aligns audio embeddings with the language model via caption-based pretraining, then adapts it for episodic in-context inference through federated optimization. At test time, given a small labeled support set, the model diagnoses an unseen query through multimodal reasoning. On held-out respiratory and cardiac conditions, FSC achieves 71.6% accuracy in 2-way 2-shot evaluation, outperforming audio-language baselines by over 9%.
