Manipulating Sparse Double Descent

Ya Shi Zhang
1/19/2024
cs.LG

No confident code implementation found for this paper.

Abstract

This paper investigates the double descent phenomenon in two-layer neural networks, focusing on the role of L1 regularization and representation dimensions. It explores an alternative double descent phenomenon, named sparse double descent. The study emphasizes the complex relationship between model complexity, sparsity, and generalization, and suggests further research into more diverse models and datasets. The findings contribute to a deeper understanding of neural network training and optimization.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Understanding the Double Descent Phenomenon in Deep Learning

87% match2024

Towards Understanding Epoch-wise Double descent in Two-layer Linear Neural Networks

85% match2024

Double Descent Demystified: Identifying, Interpreting & Ablating the Sources of a Deep Learning Puzzle

84% match1 repo712023

Unraveling the Enigma of Double Descent: An In-depth Analysis through the Lens of Learned Feature Space

84% match2026

Low-rank bias, weight decay, and model merging in neural networks

83% match2025

A U-turn on Double Descent: Rethinking Parameter Counting in Statistical Learning

83% match2026

Cite this paper

@article{zhang2024manipulating,
  title  = {Manipulating Sparse Double Descent},
  author = {Ya Shi Zhang},
  year   = {2024},
  eprint = {2401.10686},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2401.10686v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2401.10686.svg)](https://codeofpaper.com/paper/2401.10686)

Discussion