Optimizing Inference in Transformer-Based Models: A Multi-Method Benchmark

Siu Hang Ho, Prasad Ganesan, Nguyen Duong, Daniel Schlabig
9/22/2025
cs.LG

No official code found. Best implementation: ARTIFACT-CX/DiT-optimizations-benchmark (3 stars, last commit 2025-12-18).

Abstract

Efficient inference is a critical challenge in deep generative modeling, particularly as diffusion models grow in capacity and complexity. While increased complexity often improves accuracy, it raises compute costs, latency, and memory requirements. This work investigates techniques such as pruning, quantization, knowledge distillation, and simplified attention to reduce computational overhead without impacting performance. The study also explores the Mixture of Experts (MoE) approach to further enhance efficiency. These experiments provide insights into optimizing inference for the state-of-the-art Fast Diffusion Transformer (fast-DiT) model.

View on arXivView PDF

Code Implementations(1)

ARTIFACT-CX/DiT-optimizations-benchmarkCommunity65%

Optimizing Inference in Transformer-Based Models: A Multi-Method Benchmark

30Shell, PythonApr 19, 202510 months ago

Similar Papers

Exploiting Inter-Layer Expert Affinity for Accelerating Mixture-of-Experts Model Inference

86% match1 repo172024

DiffSparse: Accelerating Diffusion Transformers with Learned Token Sparsity

86% match2026

DiffPro: Joint Timestep and Layer-Wise Precision Optimization for Efficient Diffusion Inference

85% match2025

Expert Race: A Flexible Routing Strategy for Scaling Diffusion Transformer with Mixture of Experts

84% match2025

Remix-DiT: Mixing Diffusion Transformers for Multi-Expert Denoising

84% match1 repo182026

EC-DIT: Scaling Diffusion Transformers with Adaptive Expert-Choice Routing

84% match2024

Cite this paper

@article{ho2025optimizing,
  title  = {Optimizing Inference in Transformer-Based Models: A Multi-Method Benchmark},
  author = {Siu Hang Ho and Prasad Ganesan and Nguyen Duong and Daniel Schlabig},
  year   = {2025},
  eprint = {2509.17894},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2509.17894v2}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2509.17894.svg)](https://codeofpaper.com/paper/2509.17894)

Discussion