DMC4ML: Data Movement Complexity for Machine Learning

Chen Ding, Christopher Kanan, Dylan McKellips, Toranosuke Ozawa, Arian Shahmirza +1 more
12/22/2023
eess.SYcs.LG

No confident code implementation found for this paper.

Abstract

The greatest demand for today's computing is machine learning. This paper analyzes three machine learning algorithms: transformers, spatial convolution, and FFT. The analysis is novel in three aspects. First, it measures the cost of memory access on an abstract memory hierarchy, instead of traditional time or space complexity. Second, the analysis is asymptotic and identifies the primary sources of the memory cost. Finally, the result is symbolic, which can be used to select algorithmic parameters such as the group size in grouped query attention for any dimension size and number of heads and the batch size for batched convolution for any image size and kernel size.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

An Experimental Evaluation of Machine Learning Training on a Real Processing-in-Memory System

83% match1 repo292022

Fast RoPE Attention: Combining the Polynomial Method and Fast Fourier Transform

83% match2025

Can LLMs Revolutionize the Design of Explainable and Efficient TinyML Models?

83% match2025

Operation-Level Performance Benchmarking of Graph Neural Networks for Scientific Applications

82% match1 repo82022

Generating Accurate Virtual Examples For Lifelong Machine Learning

82% match2023

Embedded Machine Learning for Microcontroller-Class Edge Devices: Data, Feature, Evaluation, and Deployment Pipelines

82% match2026

Cite this paper

@article{ding2023dmcml,
  title  = {DMC4ML: Data Movement Complexity for Machine Learning},
  author = {Chen Ding and Christopher Kanan and Dylan McKellips and Toranosuke Ozawa and Arian Shahmirza and Wesley Smith},
  year   = {2023},
  eprint = {2312.14441},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2312.14441v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2312.14441.svg)](https://codeofpaper.com/paper/2312.14441)

Discussion