Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networks

Jing An, Jianfeng Lu
4/18/2023
cs.LGmath.OCstat.ML

No confident code implementation found for this paper. 2 possible matches can be shown below.

Abstract

We study the convergence of stochastic gradient descent (SGD) for non-convex objective functions. We establish the local convergence with positive probability under the local Łojasiewicz condition introduced by Chatterjee in \cite{chatterjee2022convergence} and an additional local structural assumption of the loss function landscape. A key component of our proof is to ensure that the whole trajectories of SGD stay inside the local region with a positive probability. We also provide examples of neural networks with finite widths such that our assumptions hold.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 2 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Convergence of continuous-time stochastic gradient descent with applications to deep neural networks

86% match2024

The global convergence time of stochastic gradient descent in non-convex landscapes: Sharp estimates via large deviations

85% match2025

A Local Polyak-Lojasiewicz and Descent Lemma of Gradient Descent For Overparametrized Linear Models

84% match2025

Non-convergence to the optimal risk for Adam and stochastic gradient descent optimization in the training of deep neural networks

84% match2025

Is Stochastic Gradient Descent Effective? A PDE Perspective on Machine Learning processes

83% match2025

Non-convergence to global minimizers for Adam and stochastic gradient descent optimization and constructions of local minimizers in the training of artificial neural networks

83% match2024

Cite this paper

@article{an2023convergence,
  title  = {Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networks},
  author = {Jing An and Jianfeng Lu},
  year   = {2023},
  eprint = {2304.09221},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2304.09221v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2304.09221.svg)](https://codeofpaper.com/paper/2304.09221)

Discussion