Fast Optimizer Benchmark

Simon Blauth, Tobias Bürger, Zacharias Häringer, Jörg Franke, Frank Hutter
6/26/2024
cs.LGcs.AI

Official code: automl/FOB (12 stars, Apache-2.0, last commit 2025-04-27).

Abstract

In this paper, we present the Fast Optimizer Benchmark (FOB), a tool designed for evaluating deep learning optimizers during their development. The benchmark supports tasks from multiple domains such as computer vision, natural language processing, and graph learning. The focus is on convenient usage, featuring human-readable YAML configurations, SLURM integration, and plotting utilities. FOB can be used together with existing hyperparameter optimization (HPO) tools as it handles training and resuming of runs. The modular design enables integration into custom pipelines, using it simply as a collection of tasks. We showcase an optimizer comparison as a usage example of our tool. FOB can be found on GitHub: https://github.com/automl/FOB.

View on arXivView PDF

Code Implementations(1)

automl/FOBOfficial100%
Apache-2.0
128Shell, PythonJun 4, 20241 years agoApache-2.0

Similar Papers

Development of Deep Learning Optimizers: Approaches, Concepts, and Update Rules

82% match2025

Benchmarking Deep Learning Models on NVIDIA Jetson Nano for Real-Time Systems: An Empirical Investigation

81% match2024

Efficient Deep Learning Board: Training Feedback Is Not All You Need

81% match2024

DeepCAVE: A Visualization and Analysis Tool for Automated Machine Learning

80% match2025

Fantastic Pretraining Optimizers and Where to Find Them

80% match2025

carps: A Framework for Comparing N Hyperparameter Optimizers on M Benchmarks

80% match2025

Cite this paper

@article{blauth2024fast,
  title  = {Fast Optimizer Benchmark},
  author = {Simon Blauth and Tobias Bürger and Zacharias Häringer and Jörg Franke and Frank Hutter},
  year   = {2024},
  eprint = {2406.18701},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2406.18701v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2406.18701.svg)](https://codeofpaper.com/paper/2406.18701)

Discussion