Search papers, labs, and topics across Lattice.
This paper introduces the MobenFL benchmark, a comprehensive evaluation framework for federated learning (FL) in medical imaging that integrates 20 advanced FL algorithms and 22 datasets across 12 organs. By addressing the limitations of existing benchmarks, which often focus narrowly on model accuracy and single modalities, MobenFL enables a more holistic assessment of FL's efficacy in diverse clinical scenarios. The results underscore the importance of including metrics for algorithmic efficiency and privacy protection, paving the way for more reliable clinical applications of FL in healthcare.
A unified benchmark that evaluates federated learning in medical imaging across multiple organs reveals critical gaps in existing assessments and emphasizes the need for efficiency and privacy metrics.
The privacy requirements of medical data and its substantial variations across organs and modalities hinder the clinical implementation of medical AI. Federated learning (FL) is a feasible approach to overcome these challenges. Due to the continuous emergence of FL algorithms and the highly heterogeneous nature of medical data, objectively evaluating their performance in real-world clinical settings remains difficult. Therefore, a comprehensive federated medical imaging benchmark, serving as a unified evaluation standard, is crucial for advancing the technology toward reliable clinical application. Existing federated medical imaging benchmarks have not yet adequately incorporated state-of-the-art algorithms, are limited to data from single organs or modalities, and overly emphasize model accuracy, making it difficult to comprehensively assess the overall efficacy of FL in real-world medical environments. To address these challenges, we developed the MobenFL benchmark. This benchmark integrates 20 cutting-edge FL algorithms and 22 medical imaging datasets, covering 12 critical organs across the human body, surpassing existing benchmark in breadth. In terms of evaluation dimensions, MobenFL not only assesses performance but also systematically incorporates key metrics such as algorithmic efficiency and privacy protection capabilities. Additionally, it conducts specialized evaluations for complex real-world clinical scenarios involving different diseases, devices, and imaging modalities, thereby providing a comprehensive and in-depth evaluation framework for the clinical application of FL in the medical field.
