PuSH - Publication Server of Helmholtz Zentrum München: Inverse Dirichlet weighting enables reliable training of physics informed neural networks.

Navigation

Home

Deutsch

Research

Advanced Search

Browse by ...

... Journal

... Publication Type

... Research Data

... Publication Year

Publication overview

Support & Contact

Contact persons

Help

Data protection

Maddu, S.* ; Sturm, D.* ; Müller, C.L. ; Sbalzarini, I.F.*

Inverse Dirichlet weighting enables reliable training of physics informed neural networks.

Mach. Learn.: Sci. Technol. 3:015026 (2022)

Publ. Version/Full Text

DOI

	Open Access Gold

Abstract
Metrics
Extra information

We characterize and remedy a failure mode that may arise from multi-scale dynamics with scale imbalances during training of deep neural networks, such as physics informed neural networks (PINNs). PINNs are popular machine-learning templates that allow for seamless integration of physical equation models with data. Their training amounts to solving an optimization problem over a weighted sum of data-fidelity and equation-fidelity objectives. Conflicts between objectives can arise from scale imbalances, heteroscedasticity in the data, stiffness of the physical equation, or from catastrophic interference during sequential training. We explain the training pathology arising from this and propose a simple yet effective inverse Dirichlet weighting strategy to alleviate the issue. We compare with Sobolev training of neural networks, providing the baseline of analytically epsilon-optimal training. We demonstrate the effectiveness of inverse Dirichlet weighting in various applications, including a multi-scale model of active turbulence, where we show orders of magnitude improvement in accuracy and convergence over conventional PINN training. For inverse modeling using sequential training, we find that inverse Dirichlet weighting protects a PINN against catastrophic forgetting.

Altmetric

Additional Metrics?

[➜Log in]

Edit extra informations Login

Publication type Article: Journal article

Document type Scientific Article

Corresponding Author

Keywords Physics-informed Neural Networks ; Multi-scale Modeling ; Active Turbulence ; Catastrophic Forgetting ; Multi-objective Training ; Gradient Flow Regularization; Algorithm

ISSN (print) / ISBN 2632-2153

e-ISSN 2632-2153

Journal Machine Learning: Science and Technology

Quellenangaben Volume: 3, Issue: 1, Article Number: 015026

Publisher Institute of Physics Publishing (IOP)

Publishing Place Temple Circus, Temple Way, Bristol Bs1 6be, England

Non-patent literature Publications

Reviewing status Peer reviewed

Institute(s) Institute of Computational Biology (ICB)

Grants Saxon Ministry for Science, Culture and Tourism (SMWK)
German Research Foundation (DFG)
Center for Scalable Data Analytics and Artificial Intelligence (ScaDS.AI) Dresden/Leipzig - Federal Ministry of Education and Research (BMBF)
Center for Advanced Systems Understanding (CASUS) - Germany's Federal Ministry of Education and Research (BMBF)