Ultimate tensorization: compressing convolutional and FC layers alike

Timur Garipov, Dmitry Podoprikhin, Alexander Novikov, Dmitry Vetrov
11/10/2016
cs.LG

Official code: timgaripov/TensorNet-TF (214 stars, last commit 2017-08-29).

Abstract

Convolutional neural networks excel in image recognition tasks, but this comes at the cost of high computational and memory complexity. To tackle this problem, [1] developed a tensor factorization framework to compress fully-connected layers. In this paper, we focus on compressing convolutional layers. We show that while the direct application of the tensor framework [1] to the 4-dimensional kernel of convolution does compress the layer, we can do better. We reshape the convolutional kernel into a tensor of higher order and factorize it. We combine the proposed approach with the previous work to compress both convolutional and fully-connected layers of a network and achieve 80x network compression rate with 1.1% accuracy drop on the CIFAR-10 dataset.

View on arXivView PDF

Code Implementations(1)

timgaripov/TensorNet-TFOfficial100%

TensorNet (TensorFlow implementation)

21456PythonMay 3, 20169 years ago

Similar Papers

Tensor network compressibility of convolutional models

88% match2024

Multiscale Tensor Summation Factorization as a New Neural Network Layer (MTS Layer) for Multidimensional Data Processing

88% match1 repo32025

Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks

87% match2025

Tensorization is a powerful but underexplored tool for compression and interpretability of neural networks

87% match2025

Convolutional Neural Network Compression Based on Low-Rank Decomposition

86% match2024

conv_einsum: A Framework for Representation and Fast Evaluation of Multilinear Operations in Convolutional Tensorial Neural Networks

86% match2024

Cite this paper

@article{garipov2016ultimate,
  title  = {Ultimate tensorization: compressing convolutional and FC layers alike},
  author = {Timur Garipov and Dmitry Podoprikhin and Alexander Novikov and Dmitry Vetrov},
  year   = {2016},
  eprint = {1611.03214},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1611.03214}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1611.03214.svg)](https://codeofpaper.com/paper/1611.03214)

Discussion