TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision

Zhuo Chen, Jacob McCarran, Esteban Vizcaino, Marin Soljačić, Di Luo
4/16/2024
cs.LGmath.NAphysics.comp-ph

No official code found. Best implementation: pde-sim/teng (21 stars, last commit 2024-11-18).

Abstract

Partial differential equations (PDEs) are instrumental for modeling dynamical systems in science and engineering. The advent of neural networks has initiated a significant shift in tackling these complexities though challenges in accuracy persist, especially for initial value problems. In this paper, we introduce the $\textit{Time-Evolving Natural Gradient (TENG)}$, generalizing time-dependent variational principles and optimization-based time integration, leveraging natural gradient optimization to obtain high accuracy in neural-network-based PDE solutions. Our comprehensive development includes algorithms like TENG-Euler and its high-order variants, such as TENG-Heun, tailored for enhanced precision and efficiency. TENG's effectiveness is further validated through its performance, surpassing current leading methods and achieving $\textit{machine precision}$ in step-by-step optimizations across a spectrum of PDEs, including the heat equation, Allen-Cahn equation, and Burgers' equation.

View on arXivView PDF

Code Implementations(1)

pde-sim/tengCommunity100%

Official Code for ICML 2024 paper "TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision"

216PythonJun 3, 20241 years agoNOASSERTION

Similar Papers

TENG++: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets under General Boundary Conditions

94% match2025

TENG-BC: Unified Time-Evolving Natural Gradient for Neural PDE Solvers with General Boundary Conditions

94% match2026

Fast training of accurate physics-informed neural networks without gradient descent

85% match2024

A Stable and Scalable Method for Solving Initial Value PDEs with Neural Networks

83% match1 repo112023

Low-Rank Adaptation of Evolutionary Deep Neural Networks for Efficient Learning of Time-Dependent PDEs

82% match2025

Gradient Enhanced Self-Training Physics-Informed Neural Network (gST-PINN) for Solving Nonlinear Partial Differential Equations

82% match2025

Cite this paper

@article{chen2024teng,
  title  = {TENG: Time-Evolving Natural Gradient for Solving PDEs With Deep Neural Nets Toward Machine Precision},
  author = {Zhuo Chen and Jacob McCarran and Esteban Vizcaino and Marin Soljačić and Di Luo},
  year   = {2024},
  eprint = {2404.10771},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2404.10771v2},
  journal = {ICML 2024 2024}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2404.10771.svg)](https://codeofpaper.com/paper/2404.10771)

Discussion