Learning-Rate-Free Learning: Dissecting D-Adaptation and Probabilistic Line Search

Max McGuinness
8/6/2023
cs.LGmath.OC

No confident code implementation found for this paper. 1 possible match can be shown below.

Abstract

This paper explores two recent methods for learning rate optimisation in stochastic gradient descent: D-Adaptation (arXiv:2301.07733) and probabilistic line search (arXiv:1502.02846). These approaches aim to alleviate the burden of selecting an initial learning rate by incorporating distance metrics and Gaussian process posterior estimates, respectively. In this report, I provide an intuitive overview of both methods, discuss their shared design goals, and devise scope for merging the two algorithms.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 1 weaker match is hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Training Neural Networks with Optimal Double-Bayesian Learning

88% match2026

BFE and AdaBFE: A New Approach in Learning Rate Automation for Stochastic Optimization

86% match2022

Improved Binary Forward Exploration: Learning Rate Scheduling Method for Stochastic Optimization

84% match2022

Explainable Learning Rate Regimes for Stochastic Optimization

83% match2025

Stochastic Gradient Optimization with Model-Assisted Sampling

81% match2026

AutoSGD: Automatic Learning Rate Selection for Stochastic Gradient Descent

81% match2025

Cite this paper

@article{mcguinness2023learningratefree,
  title  = {Learning-Rate-Free Learning: Dissecting D-Adaptation and Probabilistic Line Search},
  author = {Max McGuinness},
  year   = {2023},
  eprint = {2308.03102},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2308.03102v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2308.03102.svg)](https://codeofpaper.com/paper/2308.03102)

Discussion