Far-HO: A Bilevel Programming Package for Hyperparameter Optimization and Meta-Learning

Luca Franceschi, Riccardo Grazzi, Massimiliano Pontil, Saverio Salzo, Paolo Frasconi
6/13/2018
cs.MScs.LGstat.ML

Official code: lucfra/FAR-HO (191 stars, MIT, TensorFlow, last commit 2020-03-24).

Abstract

In (Franceschi et al., 2018) we proposed a unified mathematical framework, grounded on bilevel programming, that encompasses gradient-based hyperparameter optimization and meta-learning. We formulated an approximate version of the problem where the inner objective is solved iteratively, and gave sufficient conditions ensuring convergence to the exact problem. In this work we show how to optimize learning rates, automatically weight the loss of single examples and learn hyper-representations with Far-HO, a software package based on the popular deep learning framework TensorFlow that allows to seamlessly tackle both HO and ML problems.

View on arXivView PDF

Code Implementations(1)

lucfra/FAR-HOOfficial100%
TensorFlowMIT

Gradient based hyperparameter optimization & meta-learning package for TensorFlow

19148Python, Jupyter NotebookDec 4, 20176 years agoMIT
gradient-descenthyperparameter-optimizationoptimizationtensorflow

Similar Papers

Newton-Based Methods for Bilevel Optimization Beyond Lower-Level Strong Convexity

89% match2026

Penalty Method for Inversion-Free Deep Bilevel Optimization

86% match1 repo152019

BiERL: A Meta Evolutionary Reinforcement Learning Framework via Bilevel Optimization

86% match1 repo52023

Overcoming Lower-Level Constraints in Bilevel Optimization: A Novel Approach with Regularized Gap Functions

85% match1 repo2024

Analyzing Inexact Hypergradients for Bilevel Learning

85% match2023

MADA: Meta-Adaptive Optimizers through hyper-gradient Descent

84% match1 repo52024

Cite this paper

@article{franceschi2018farho,
  title  = {Far-HO: A Bilevel Programming Package for Hyperparameter Optimization and Meta-Learning},
  author = {Luca Franceschi and Riccardo Grazzi and Massimiliano Pontil and Saverio Salzo and Paolo Frasconi},
  year   = {2018},
  eprint = {1806.04941},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1806.04941}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1806.04941.svg)](https://codeofpaper.com/paper/1806.04941)

Discussion