Parameter-free version of Adaptive Gradient Methods for Strongly-Convex Functions

Deepak Gouda, Hassan Naveed, Salil Kamath
6/11/2023
cs.LGmath.OC

No confident code implementation found for this paper.

Abstract

The optimal learning rate for adaptive gradient methods applied to λ-strongly convex functions relies on the parameters λ and learning rate η. In this paper, we adapt a universal algorithm along the lines of Metagrad, to get rid of this dependence on λ and η. The main idea is to concurrently run multiple experts and combine their predictions to a master algorithm. This master enjoys O(d log T) regret bounds.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Adaptive proximal gradient methods are universal without approximation

83% match1 repo42024

Towards Simple and Provable Parameter-Free Adaptive Gradient Methods

82% match1 repo2024

Unifying Nesterov's Accelerated Gradient Methods for Convex and Strongly Convex Objective Functions

82% match1 repo2026

Dual Adaptivity: Universal Algorithms for Minimizing the Adaptive Regret of Convex Functions

82% match2025

Interpreting Adaptive Gradient Methods by Parameter Scaling for Learning-Rate-Free Optimization

82% match2024

Improving Adaptive Online Learning Using Refined Discretization

81% match2023

Cite this paper

@article{gouda2023parameterfree,
  title  = {Parameter-free version of Adaptive Gradient Methods for Strongly-Convex Functions},
  author = {Deepak Gouda and Hassan Naveed and Salil Kamath},
  year   = {2023},
  eprint = {2306.06613},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2306.06613v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2306.06613.svg)](https://codeofpaper.com/paper/2306.06613)

Discussion