A convergence result of a continuous model of deep learning via Łojasiewicz--Simon inequality

Noboru Isobe
11/26/2023
cs.LGmath.APmath.FA

No confident code implementation found for this paper.

Abstract

This study focuses on a Wasserstein-type gradient flow, which represents an optimization process of a continuous model of a Deep Neural Network (DNN). First, we establish the existence of a minimizer for an average loss of the model under $L^2$-regularization. Subsequently, we show the existence of a curve of maximal slope of the loss. Our main result is the convergence of flow to a critical point of the loss as time goes to infinity. An essential aspect of proving this result involves the establishment of the Łojasiewicz--Simon gradient inequality for the loss. We derive this inequality by assuming the analyticity of NNs and loss functions. Our proofs offer a new approach for analyzing the asymptotic behavior of Wasserstein-type gradient flows for nonconvex functionals.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Convergence beyond the over-parameterized regime using Rayleigh quotients

85% match2023

Understanding the training of infinitely deep and wide ResNets with Conditional Optimal Transport

84% match2024

Gradient Flow Convergence Guarantee for General Neural Network Architectures

83% match2025

SAD Neural Networks: Divergent Gradient Flows and Asymptotic Optimality via o-minimal Structures

82% match1 repo2025

Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Loss

82% match2023

Gradient Flow Equations for Deep Linear Neural Networks: A Survey from a Network Perspective

82% match2025

Cite this paper

@article{isobe2023convergence,
  title  = {A convergence result of a continuous model of deep learning via Łojasiewicz--Simon inequality},
  author = {Noboru Isobe},
  year   = {2023},
  eprint = {2311.15365},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2311.15365v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2311.15365.svg)](https://codeofpaper.com/paper/2311.15365)

Discussion