Till the Layers Collapse: Compressing a Deep Neural Network through the Lenses of Batch Normalization Layers

Zhu Liao, Nour Hezbri, Victor Quétu, Van-Tam Nguyen, Enzo Tartaglione
12/19/2024
cs.LGcs.CLcs.CV

No official code found. Best implementation: ZhuLIAO001/TLC (5 stars, last commit 2025-05-02).

Abstract

Today, deep neural networks are widely used since they can handle a variety of complex tasks. Their generality makes them very powerful tools in modern technology. However, deep neural networks are often overparameterized. The usage of these large models consumes a lot of computation resources. In this paper, we introduce a method called \textbf{T}ill the \textbf{L}ayers \textbf{C}ollapse (TLC), which compresses deep neural networks through the lenses of batch normalization layers. By reducing the depth of these networks, our method decreases deep neural networks' computational requirements and overall latency. We validate our method on popular models such as Swin-T, MobileNet-V2, and RoBERTa, across both image classification and natural language processing (NLP) tasks.

View on arXivView PDF

Code Implementations(1)

ZhuLIAO001/TLCCommunity100%

Here we provide the code for the paper 'Till the Layers Collapse: Compressing a Deep Neural Network through the Lenses of Batch Normalization Layers.'

50PythonDec 13, 20241 years ago

Similar Papers

LayerMerge: Neural Network Depth Compression through Layer Pruning and Merging

82% match1 repo322024

Speeding Up Image Classifiers with Little Companions

82% match2024

1 bit is all we need: binary normalized neural networks

82% match2025

Tensor Decomposition for Model Reduction in Neural Networks: A Review

82% match2023

Towards Understanding Neural Collapse: The Effects of Batch Normalization and Weight Decay

82% match2023

Step by Step Network

81% match2025

Cite this paper

@article{liao2024till,
  title  = {Till the Layers Collapse: Compressing a Deep Neural Network through the Lenses of Batch Normalization Layers},
  author = {Zhu Liao and Nour Hezbri and Victor Quétu and Van-Tam Nguyen and Enzo Tartaglione},
  year   = {2024},
  eprint = {2412.15077},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2412.15077v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2412.15077.svg)](https://codeofpaper.com/paper/2412.15077)

Discussion