Accelerating the Low-Rank Decomposed Models

Habib Hajimolahoseini, Walid Ahmed, Austin Wen, Yang Liu
7/24/2024
cs.LGcs.CL

No confident code implementation found for this paper. 5 possible matches can be shown below.

Abstract

Tensor decomposition is a mathematically supported technique for data compression. It consists of applying some kind of a Low Rank Decomposition technique on the tensors or matrices in order to reduce the redundancy of the data. However, it is not a popular technique for compressing the AI models duo to the high number of new layers added to the architecture after decomposition. Although the number of parameters could shrink significantly, it could result in the model be more than twice deeper which could add some latency to the training or inference. In this paper, we present a comprehensive study about how to modify low rank decomposition technique in AI models so that we could benefit from both high accuracy and low memory consumption as well as speeding up the training and inference

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 5 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Training Acceleration of Low-Rank Decomposed Networks using Sequential Freezing and Rank Quantization

89% match2023

Convolutional Neural Network Compression Based on Low-Rank Decomposition

86% match2024

Tensor Train Low-rank Approximation (TT-LoRA): Democratizing AI with Accelerated LLMs

84% match2024

Memory-Efficient Vision Transformers: An Activation-Aware Mixed-Rank Compression Strategy

84% match2024

An Adaptive Tensor-Train Decomposition Approach for Efficient Deep Neural Network Compression

83% match2024

Low-Rank Tensor Approximation of Weights in Large Language Models via Cosine Lanczos Bidiagonalization

83% match2026

Cite this paper

@article{hajimolahoseini2024accelerating,
  title  = {Accelerating the Low-Rank Decomposed Models},
  author = {Habib Hajimolahoseini and Walid Ahmed and Austin Wen and Yang Liu},
  year   = {2024},
  eprint = {2407.20266},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2407.20266v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2407.20266.svg)](https://codeofpaper.com/paper/2407.20266)

Discussion