SCoTTi: Save Computation at Training Time with an adaptive framework

Ziyu Lin, Enzo Tartaglione, Van-Tam Nguyen
12/19/2023
cs.LGcs.AIcs.CV

No official code found. Best implementation: liziyu403/SCoTTi-Save-Computation-at-Training-Time-with-an-adaptive-framework (1 star, MIT, last commit 2024-06-29).

Abstract

On-device training is an emerging approach in machine learning where models are trained on edge devices, aiming to enhance privacy protection and real-time performance. However, edge devices typically possess restricted computational power and resources, making it challenging to perform computationally intensive model training tasks. Consequently, reducing resource consumption during training has become a pressing concern in this field. To this end, we propose SCoTTi (Save Computation at Training Time), an adaptive framework that addresses the aforementioned challenge. It leverages an optimizable threshold parameter to effectively reduce the number of neuron updates during training which corresponds to a decrease in memory and computation footprint. Our proposed approach demonstrates superior performance compared to the state-of-the-art methods regarding computational resource savings on various commonly employed benchmarks and popular architectures, including ResNets, MobileNet, and Swin-T.

View on arXivView PDF

Code Implementations(1)

liziyu403/SCoTTi-Save-Computation-at-Training-Time-with-an-adaptive-frameworkCommunity100%
MIT

we propose SCoTTi (Save Computation at Training Time), an adaptive framework that addresses the aforementioned challenge. It leverages an optimizable threshold parameter to effectively reduce the number of neuron updates during training which corresponds to a decrease in memory and computation footprint.

10PythonAug 14, 20232 years agoMIT

Similar Papers

Tin-Tin: Towards Tiny Learning on Tiny Devices with Integer-based Neural Network Training

86% match2025

ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsity

85% match2022

EMO: Edge Model Overlays to Scale Model Size in Federated Learning

85% match2025

Towards On-device Learning on the Edge: Ways to Select Neurons to Update under a Budget Constraint

85% match1 repo32023

POET: Training Neural Networks on Tiny Devices with Integrated Rematerialization and Paging

85% match1 repo1692022

Developing a Resource-Constraint EdgeAI model for Surface Defect Detection

84% match2023

Cite this paper

@article{lin2023scotti,
  title  = {SCoTTi: Save Computation at Training Time with an adaptive framework},
  author = {Ziyu Lin and Enzo Tartaglione and Van-Tam Nguyen},
  year   = {2023},
  eprint = {2312.12483},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2312.12483v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2312.12483.svg)](https://codeofpaper.com/paper/2312.12483)

Discussion