On-device AI: Quantization-aware Training of Transformers in Time-Series

Tianheng Ling, Gregor Schiele
8/29/2024
cs.LGcs.AI

No confident code implementation found for this paper. 3 possible matches can be shown below.

Abstract

Artificial Intelligence (AI) models for time-series in pervasive computing keep getting larger and more complicated. The Transformer model is by far the most compelling of these AI models. However, it is difficult to obtain the desired performance when deploying such a massive model on a sensor device with limited resources. My research focuses on optimizing the Transformer model for time-series forecasting tasks. The optimized model will be deployed as hardware accelerators on embedded Field Programmable Gate Arrays (FPGAs). I will investigate the impact of applying Quantization-aware Training to the Transformer model to reduce its size and runtime memory footprint while maximizing the advantages of FPGAs.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. 3 weaker matches are hidden by default — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Integer-only Quantized Transformers for Embedded FPGA-based Time-series Forecasting in AIoT

89% match1 repo92024

Ultra Memory-Efficient On-FPGA Training of Transformers via Tensor-Compressed Optimization

83% match2025

Survey and Taxonomy: The Role of Data-Centric AI in Transformer-Based Time Series Forecasting

83% match2024

Energy-Efficient Transformer Inference: Optimization Strategies for Time Series Classification

83% match2025

TQ-DiT: Efficient Time-Aware Quantization for Diffusion Transformers

82% match2025

QT-TDM: Planning With Transformer Dynamics Model and Autoregressive Q-Learning

82% match2 repos62024

Cite this paper

@article{ling2024ondevice,
  title  = {On-device AI: Quantization-aware Training of Transformers in Time-Series},
  author = {Tianheng Ling and Gregor Schiele},
  year   = {2024},
  eprint = {2408.16495},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2408.16495v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2408.16495.svg)](https://codeofpaper.com/paper/2408.16495)

Discussion