MUSCO: Multi-Stage Compression of neural networks

Julia Gusak, Maksym Kholiavchenko, Evgeny Ponomarev, Larisa Markeeva, Ivan Oseledets +1 more
3/24/2019
cs.LGcs.CVstat.ML

Official code: musco-ai/musco-pytorch (72 stars, BSD-3-Clause, PyTorch, last commit 2021-02-16).

Abstract

The low-rank tensor approximation is very promising for the compression of deep neural networks. We propose a new simple and efficient iterative approach, which alternates low-rank factorization with a smart rank selection and fine-tuning. We demonstrate the efficiency of our method comparing to non-iterative ones. Our approach improves the compression rate while maintaining the accuracy for a variety of tasks.

View on arXivView PDF

Code Implementations(1)

musco-ai/musco-pytorchOfficial100%
PyTorchBSD-3-Clause

MUSCO: MUlti-Stage COmpression of neural networks

7216Python, Jupyter NotebookMay 8, 20195 years agoBSD-3-Clause
cp-decompositiondeep-neural-networkslow-rankmodel-accelerationmodel-compression+7 more

Similar Papers

Low-Rank Tensor Decompositions for the Theory of Neural Networks

88% match2025

Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Training

87% match2023

Convolutional neural networks with low-rank regularization

87% match1 repo962015

CompressNAS : A Fast and Efficient Technique for Model Compression using Decomposition

87% match2025

An Adaptive Tensor-Train Decomposition Approach for Efficient Deep Neural Network Compression

87% match2024

Theoretical Guarantees for Low-Rank Compression of Deep Neural Networks

86% match2025

Cite this paper

@article{gusak2019musco,
  title  = {MUSCO: Multi-Stage Compression of neural networks},
  author = {Julia Gusak and Maksym Kholiavchenko and Evgeny Ponomarev and Larisa Markeeva and Ivan Oseledets and Andrzej Cichocki},
  year   = {2019},
  eprint = {1903.09973},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1903.09973}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1903.09973.svg)](https://codeofpaper.com/paper/1903.09973)

Discussion