TY  - JOUR
AB  - Federated learning (FL) is an evolving machine learning technique that allows collaborative model training without sharing the original data among participants. In real-world scenarios, data residing at multiple clients are often heterogeneous in terms of different resolutions, magnifications, scanners, or imaging protocols, and thus challenging for global FL model convergence in collaborative training. Most of the existing FL methods consider data heterogeneity within one domain by assuming same data variation in each client site. In this paper, we consider data heterogeneity in FL with different domains of heterogeneous data by raising the problems of domain-shift, class-imbalance, and missing data. We propose a method, multi-domain FL as a solution to heterogeneous training data from multiple domains by training robust vision transformer model. We use two loss functions, one for correctly predicting class labels and other for encouraging similarity and dissimilarity over latent features, to optimize the global FL model. We perform various experiments using different convolution-based networks and non-convolutional Transformer architectures on multi-domain datasets. We evaluate the proposed approach on benchmark datasets and compare with the existing FL methods. Our results show the superiority of the proposed approach which performs better in term of robust FL global model than the exiting methods.
AU  - Ahmad Madni, H.*
AU  - Umer, R.M.
AU  - Luca Foresti, G.*
C1  - 70984
C2  - 55853
CY  - Temple Circus, Temple Way, Bristol Bs1 6be, England
TI  - Exploiting data diversity in multi-domain federated learning.
JO  - Mach. Learn.: Sci. Technol.
VL  - 5
IS  - 2
PB  - Iop Publishing Ltd
PY  - 2024
SN  - 2632-2153
ER  - 

TY  - JOUR
AB  - We characterize and remedy a failure mode that may arise from multi-scale dynamics with scale imbalances during training of deep neural networks, such as physics informed neural networks (PINNs). PINNs are popular machine-learning templates that allow for seamless integration of physical equation models with data. Their training amounts to solving an optimization problem over a weighted sum of data-fidelity and equation-fidelity objectives. Conflicts between objectives can arise from scale imbalances, heteroscedasticity in the data, stiffness of the physical equation, or from catastrophic interference during sequential training. We explain the training pathology arising from this and propose a simple yet effective inverse Dirichlet weighting strategy to alleviate the issue. We compare with Sobolev training of neural networks, providing the baseline of analytically epsilon-optimal training. We demonstrate the effectiveness of inverse Dirichlet weighting in various applications, including a multi-scale model of active turbulence, where we show orders of magnitude improvement in accuracy and convergence over conventional PINN training. For inverse modeling using sequential training, we find that inverse Dirichlet weighting protects a PINN against catastrophic forgetting.
AU  - Maddu, S.*
AU  - Sturm, D.*
AU  - Müller, C.L.
AU  - Sbalzarini, I.F.*
C1  - 64474
C2  - 52062
CY  - Temple Circus, Temple Way, Bristol Bs1 6be, England
TI  - Inverse Dirichlet weighting enables reliable training of physics informed neural networks.
JO  - Mach. Learn.: Sci. Technol.
VL  - 3
IS  - 1
PB  - Iop Publishing Ltd
PY  - 2022
SN  - 2632-2153
ER  -