Microscaling Data Formats for Deep Learning

Bita Darvish Rouhani, Ritchie Zhao, Ankit More, Mathew Hall, Alireza Khodamoradi +28 more
10/16/2023
cs.LGcs.AI

Official code: microsoft/microxcaling (358 stars, MIT, last commit 2025-06-18).

Abstract

Narrow bit-width data formats are key to reducing the computational and storage costs of modern deep learning applications. This paper evaluates Microscaling (MX) data formats that combine a per-block scaling factor with narrow floating-point and integer types for individual elements. MX formats balance the competing needs of hardware efficiency, model accuracy, and user friction. Empirical results on over two dozen benchmarks demonstrate practicality of MX data formats as a drop-in replacement for baseline FP32 for AI inference and training with low user friction. We also show the first instance of training generative language models at sub-8-bit weights, activations, and gradients with minimal accuracy loss and no modifications to the training recipe.

View on arXivView PDF

Code Implementations(1)

microsoft/microxcalingOfficial100%
MIT

PyTorch emulation library for Microscaling (MX)-compatible data formats

35850C++, CudaSep 5, 20231 years agoMIT

Similar Papers

Nanoscaling Floating-Point (NxFP): NanoMantissa, Adaptive Microexponents, and Code Recycling for Direct-Cast Compression of Large Language Models

84% match2024

Characterization and Mitigation of Training Instabilities in Microscaling Formats

84% match1 repo32025

INT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formats

83% match2025

Recipes for Pre-training LLMs with MXFP8

83% match2025

Is Finer Better? The Limits of Microscaling Formats in Large Language Models

82% match2026

Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats

82% match2026

Cite this paper

@article{rouhani2023microscaling,
  title  = {Microscaling Data Formats for Deep Learning},
  author = {Bita Darvish Rouhani and Ritchie Zhao and Ankit More and Mathew Hall and Alireza Khodamoradi and Summer Deng and Dhruv Choudhary and Marius Cornea and Eric Dellinger and Kristof Denolf and Stosic Dusan and Venmugil Elango and Maximilian Golub and Alexander Heinecke and Phil James-Roxby and Dharmesh Jani and Gaurav Kolhe and Martin Langhammer and Ada Li and Levi Melnick and Maral Mesmakhosroshahi and Andres Rodriguez and Michael Schulte and Rasoul Shafipour and Lei Shao and Michael Siu and Pradeep Dubey and Paulius Micikevicius and Maxim Naumov and Colin Verrilli and Ralph Wittig and Doug Burger and Eric Chung},
  year   = {2023},
  eprint = {2310.10537},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2310.10537v3}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2310.10537.svg)](https://codeofpaper.com/paper/2310.10537)

Discussion