Low Latency Transformer Inference on FPGAs for Physics Applications with hls4ml

Zhixing Jiang, Dennis Yin, Yihui Chen, Elham E Khoda, Scott Hauck +5 more
9/8/2024
cs.LG

No confident code implementation found for this paper.

Abstract

This study presents an efficient implementation of transformer architectures in Field-Programmable Gate Arrays(FPGAs) using hls4ml. We demonstrate the strategy for implementing the multi-head attention, softmax, and normalization layer and evaluate three distinct models. Their deployment on VU13P FPGA chip achieved latency less than 2us, demonstrating the potential for real-time applications. HLS4ML compatibility with any TensorFlow-built transformer model further enhances the scalability and applicability of this work. Index Terms: FPGAs, machine learning, transformers, high energy physics, LIGO

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

Sub-microsecond Transformers for Jet Tagging on FPGAs

87% match2025

HLSTransform: Energy-Efficient Llama 2 Inference on FPGAs Via High Level Synthesis

87% match1 repo1242024

Ultra Fast Transformers on FPGAs for Particle Physics Experiments

87% match2024

The Feasibility of Implementing Large-Scale Transformers on Multi-FPGA Platforms

85% match2024

Design and Implementation of an FPGA-Based Hardware Accelerator for Transformer

83% match1 repo382025

Communication-Efficient Multi-Device Inference Acceleration for Transformer Models

83% match1 repo32025

Cite this paper

@article{jiang2024latency,
  title  = {Low Latency Transformer Inference on FPGAs for Physics Applications with hls4ml},
  author = {Zhixing Jiang and Dennis Yin and Yihui Chen and Elham E Khoda and Scott Hauck and Shih-Chieh Hsu and Ekaterina Govorkova and Philip Harris and Vladimir Loncar and Eric A. Moreno},
  year   = {2024},
  eprint = {2409.05207},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2409.05207v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2409.05207.svg)](https://codeofpaper.com/paper/2409.05207)

Discussion