A multilevel approach to accelerate the training of Transformers

Guillaume Lauga, Maël Chaumette, Edgar Desainte-Maréville, Étienne Lasalle, Arthur Lebeurrier
4/24/2025
cs.LGcs.AImath.OC

No confident code implementation found for this paper.

Abstract

In this article, we investigate the potential of multilevel approaches to accelerate the training of transformer architectures. Using an ordinary differential equation (ODE) interpretation of these architectures, we propose an appropriate way of varying the discretization of these ODE Transformers in order to accelerate the training. We validate our approach experimentally by a comparison with the standard training procedure.

View on arXivView PDF

Code Implementations

No confident code match yet

We couldn't find an author-owned or strongly-evidenced community implementation for this paper. Any repos shown below are weak matches — verify before relying on them.

No code implementations found yet.

Know of an implementation? Let us know in the comments below!

Similar Papers

A Survey on Efficient Training of Transformers

85% match2023

Learning to Grow Pretrained Models for Efficient Transformer Training

85% match1 repo922026

Layer-Parallel Training for Transformers

84% match2026

A Multi-Level Framework for Accelerating Training Transformer Models

81% match1 repo72024

TripLe: Revisiting Pretrained Model Reuse and Progressive Learning for Efficient Vision Transformer Scaling and Searching

81% match2026

Early Transformers: A study on Efficient Training of Transformer Models through Early-Bird Lottery Tickets

80% match2024

Cite this paper

@article{lauga2025multilevel,
  title  = {A multilevel approach to accelerate the training of Transformers},
  author = {Guillaume Lauga and Maël Chaumette and Edgar Desainte-Maréville and Étienne Lasalle and Arthur Lebeurrier},
  year   = {2025},
  eprint = {2504.18590},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2504.18590v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2504.18590.svg)](https://codeofpaper.com/paper/2504.18590)

Discussion