Speeding up Resnet Architecture with Layers Targeted Low Rank Decomposition

Walid Ahmed, Habib Hajimolahoseini, Austin Wen, Yang Liu
9/21/2023
cs.CVcs.LG

No confident code implementation found for this paper.

Abstract

Compression of a neural network can help in speeding up both the training and the inference of the network. In this research, we study applying compression using low rank decomposition on network layers. Our research demonstrates that to acquire a speed up, the compression methodology should be aware of the underlying hardware as analysis should be done to choose which layers to compress. The advantage of our approach is demonstrated via a case study of compressing ResNet50 and training on full ImageNet-ILSVRC2012. We tested on two different hardware systems Nvidia V100 and Huawei Ascend910. With hardware targeted compression, results on Ascend910 showed 5.36% training speedup and 15.79% inference speed on Ascend310 with only 1% drop in accuracy compared to the original uncompressed model

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

HALOC: Hardware-Aware Automatic Low-Rank Compression for Compact Neural Networks

86% match2023

Training Acceleration of Low-Rank Decomposed Networks using Sequential Freezing and Rank Quantization

83% match2023

Coding for Computation: Efficient Compression of Neural Networks for Reconfigurable Hardware

83% match2025

Low Rank Optimization for Efficient Deep Learning: Making A Balance between Compact Architecture and Fast Training

83% match2023

Dynamical Low-Rank Compression of Neural Networks with Robustness under Adversarial Attacks

82% match2025

Compression of Recurrent Neural Networks using Matrix Factorization

82% match2023

Cite this paper

@article{ahmed2023speeding,
  title  = {Speeding up Resnet Architecture with Layers Targeted Low Rank Decomposition},
  author = {Walid Ahmed and Habib Hajimolahoseini and Austin Wen and Yang Liu},
  year   = {2023},
  eprint = {2309.12412},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2309.12412v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2309.12412.svg)](https://codeofpaper.com/paper/2309.12412)

Discussion