Convergence of continuous-time stochastic gradient descent with applications to deep neural networks

Gabor Lugosi, Eulalia Nualart
9/11/2024
cs.LGmath.OCstat.ML

No confident code implementation found for this paper. 2 possible matches can be shown below.

Abstract

We study a continuous-time approximation of the stochastic gradient descent process for minimizing the population expected loss in learning problems. The main results establish general sufficient conditions for the convergence, extending the results of Chatterjee (2022) established for (nonstochastic) gradient descent. We show how the main result can be applied to the case of overparametrized neural network training.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 2 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Convergence of stochastic gradient descent under a local Lojasiewicz condition for deep neural networks

86% match2023

Towards Continuous-Time Approximations for Stochastic Gradient Descent without Replacement

85% match2025

Is Learning in Biological Neural Networks based on Stochastic Gradient Descent? An analysis using stochastic processes

85% match2023

Leveraging the two timescale regime to demonstrate convergence of neural networks

84% match2023

The global convergence time of stochastic gradient descent in non-convex landscapes: Sharp estimates via large deviations

84% match2025

Is Stochastic Gradient Descent Effective? A PDE Perspective on Machine Learning processes

84% match2025

Cite this paper

@article{lugosi2024convergence,
  title  = {Convergence of continuous-time stochastic gradient descent with applications to deep neural networks},
  author = {Gabor Lugosi and Eulalia Nualart},
  year   = {2024},
  eprint = {2409.07401},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2409.07401v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2409.07401.svg)](https://codeofpaper.com/paper/2409.07401)

Discussion