Exploring Generalization in Deep Learning

Behnam Neyshabur, Srinadh Bhojanapalli, David McAllester, Nathan Srebro
6/27/2017
cs.LG

Official code: bneyshabur/generalization-bounds (35 stars, last commit 2018-12-13).

Abstract

With a goal of understanding what drives generalization in deep networks, we consider several recently suggested explanations, including norm-based control, sharpness and robustness. We study how these measures can ensure generalization, highlighting the importance of scale normalization, and making a connection between sharpness and PAC-Bayes theory. We then investigate how well the measures explain different observed phenomena.

View on arXivView PDF

Code Implementations(1)

bneyshabur/generalization-boundsOfficial100%

Computing various measures and generalization bounds on convolutional and fully connected networks

3510PythonNov 26, 20187 years ago

Similar Papers

A Modern Look at the Relationship between Sharpness and Generalization

85% match1 repo442023

Understanding Generalization through Visualizations

84% match1 repo632019

An In-depth Investigation of Sparse Rate Reduction in Transformer-like Models

83% match2024

Towards Understanding the Role of Over-Parametrization in Generalization of Neural Networks

83% match1 repo502018

Sharpness Minimization Algorithms Do Not Only Minimize Sharpness To Achieve Better Generalization

83% match2023

Predicting the Generalization Gap in Deep Networks with Margin Distributions

82% match1 repo39.0k2018

Cite this paper

@article{neyshabur2017exploring,
  title  = {Exploring Generalization in Deep Learning},
  author = {Behnam Neyshabur and Srinadh Bhojanapalli and David McAllester and Nathan Srebro},
  year   = {2017},
  eprint = {1706.08947},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1706.08947}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1706.08947.svg)](https://codeofpaper.com/paper/1706.08947)

Discussion