PuSH - Publikationsserver des Helmholtz Zentrums München

Export: TXT Text RIS Endnote (RIS) BIB BibTeX
Maddu, S.* ; Sturm, D.* ; Müller, C.L. ; Sbalzarini, I.F.*

Inverse Dirichlet weighting enables reliable training of physics informed neural networks.

Mach. Learn.: Sci. Technol. 3:015026 (2022)
Verlagsversion DOI
Open Access Gold
Creative Commons Lizenzvertrag
We characterize and remedy a failure mode that may arise from multi-scale dynamics with scale imbalances during training of deep neural networks, such as physics informed neural networks (PINNs). PINNs are popular machine-learning templates that allow for seamless integration of physical equation models with data. Their training amounts to solving an optimization problem over a weighted sum of data-fidelity and equation-fidelity objectives. Conflicts between objectives can arise from scale imbalances, heteroscedasticity in the data, stiffness of the physical equation, or from catastrophic interference during sequential training. We explain the training pathology arising from this and propose a simple yet effective inverse Dirichlet weighting strategy to alleviate the issue. We compare with Sobolev training of neural networks, providing the baseline of analytically epsilon-optimal training. We demonstrate the effectiveness of inverse Dirichlet weighting in various applications, including a multi-scale model of active turbulence, where we show orders of magnitude improvement in accuracy and convergence over conventional PINN training. For inverse modeling using sequential training, we find that inverse Dirichlet weighting protects a PINN against catastrophic forgetting.
Impact Factor
Scopus SNIP
Altmetric
6.013
0.000
Tags
Anmerkungen
Besondere Publikation
Auf Hompepage verbergern

Zusatzinfos bearbeiten
Eigene Tags bearbeiten
Privat
Eigene Anmerkung bearbeiten
Privat
Auf Publikationslisten für
Homepage nicht anzeigen
Als besondere Publikation
markieren
Publikationstyp Artikel: Journalartikel
Dokumenttyp Wissenschaftlicher Artikel
Schlagwörter Physics-informed Neural Networks ; Multi-scale Modeling ; Active Turbulence ; Catastrophic Forgetting ; Multi-objective Training ; Gradient Flow Regularization; Algorithm
Sprache englisch
Veröffentlichungsjahr 2022
HGF-Berichtsjahr 2022
ISSN (print) / ISBN 2632-2153
e-ISSN 2632-2153
Zeitschrift Machine Learning: Science and Technology
Quellenangaben Band: 3, Heft: 1, Seiten: , Artikelnummer: 015026 Supplement: ,
Verlag Institute of Physics Publishing (IOP)
Verlagsort Temple Circus, Temple Way, Bristol Bs1 6be, England
Begutachtungsstatus Peer reviewed
Institut(e) Institute of Computational Biology (ICB)
POF Topic(s) 30205 - Bioengineering and Digital Health
Forschungsfeld(er) Enabling and Novel Technologies
PSP-Element(e) G-503800-001
Förderungen Saxon Ministry for Science, Culture and Tourism (SMWK)
German Research Foundation (DFG)
Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig - Federal Ministry of Education and Research (BMBF)
Center for Advanced Systems Understanding (CASUS) - Germany's Federal Ministry of Education and Research (BMBF)
DOI 10.1088/2632-2153/ac3712
WOS ID WOS:000755293800001
Scopus ID 85126707714
Erfassungsdatum 2022-06-01
[➜Korrektur melden]