Computational competitions are the standard for benchmarking medical image analysis algorithms, but they typically use small curated test datasets acquired at a few centers, leaving a gap to the reality of diverse multicentric patient data. To this end, the Federated Tumor Segmentation (FeTS) Challenge represents the paradigm for real-world algorithmic performance evaluation. The FeTS challenge is a competition to benchmark (i) federated learning aggregation algorithms and (ii) state-of-the-art segmentation algorithms, across multiple international sites. Weight aggregation and client selection techniques were compared using a multicentric brain tumor dataset in realistic federated learning simulations, yielding benefits for adaptive weight aggregation, and efficiency gains through client sampling. Quantitative performance evaluation of state-of-the-art segmentation algorithms on data distributed internationally across 32 institutions yielded good generalization on average, albeit the worst-case performance revealed data-specific modes of failure. Similar multi-site setups can help validate the real-world utility of healthcare AI algorithms in the future.
Institut(e)Institute for Tissue Engineering and Regenerative Medicine (ITERM)
FörderungenCRUK Ministry of Health of the Czech Republic Clinical and Translational Imaging Lab at LUMS Higher Education Commission of Pakistan as part of the National Center for Big Data and Cloud Computing National Cancer Institute Herbert and Florence Irving/the Irving Trust NIH/NCATS, a research grant from Varian Medical Systems (Palo Alto, CA, USA) NIH/NIBIB Silesian University of Technology Silesian University of Technology funds through the Excellence Initiative-Research University program Canadian Institutes of Health Research (CIHR Project) CSC-Puhti supercomputer Business Finland Hong Kong Research Grants Council Juan de la Cierva fellowship National Institutes of Health (NIH) Ministry of Education, Youth and Sports of the Czech Republic Wellcome award Cancer Research UK Leeds Hospitals Charity Canada CIFAR AI Chairs Program NSF Convergence Accelerator - Track D: ImagiQ: Asynchronous and Decentralized Federated Learning for Medical Imaging ANID-Basal proyects CNPq National Institutes of Health NSF Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) CCSG Grant MH CZ - DRO (FNBr) Helmholtz Association (HA) within the project "Trustworthy Federated Data Analytics" (TFDA)