MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI

Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu +17 more
2/14/2026

No official code found. Best implementation: MMMU-Benchmark/MMMU (589 stars, Apache-2.0, last commit 2026-02-12).

Abstract

We introduce MMMU: a new benchmark designed to evaluate multimodal models on massive multi-discipline tasks demanding college-level subject knowledge and deliberate reasoning. MMMU includes 11.5K meticulously collected multimodal questions from college exams, quizzes, and text-books, covering six core disciplines: Art & Design, Busi-ness, Science, Health & Medicine, Humanities & Social Science, and Tech & Engineering. These questions span 30 subjects and 183 subfields, comprising 30 highly het-erogeneous image types, such as charts, diagrams, maps, tables, music sheets, and chemical structures. Unlike existing benchmarks, MMMU focuses on advanced perception and reasoning with domain-specific knowledge, challenging models to perform tasks akin to those faced by experts. The evaluation of 28 open-source LMMs as well as the propri-etary GPT-4V(ision) and Gemini highlights the substantial challenges posed by MMMU. Even the advanced GPT-4V and Gemini Ultra only achieve accuracies of 56% and 59% respectively, indicating significant room for improvement. We believe MMMU will stimulate the community to build next-generation multimodal foundation models towards expert artificial general intelligence.

View on arXivView PDF

Code Implementations(1)

MMMU-Benchmark/MMMUCommunity100%
Apache-2.0

This repo contains evaluation code for the paper "MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI"

58955HTML, ShellNov 23, 20238 months agoApache-2.0
computer-visiondeep-learningdeep-neural-networksevaluationfoundation-models+13 more

Similar Papers

MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models

89% match1 repo732025

BLINK: Multimodal Large Language Models Can See but Not Perceive

87% match1 repo1702026

C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models

87% match1 repo1.9k2026

SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretation

86% match1 repo132026

MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

86% match1 repo1192026

Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmark

86% match2 repos692026

Cite this paper

@article{yue2026mmmu,
  title  = {MMMU: A Massive Multi-Discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI},
  author = {Xiang Yue and Yuansheng Ni and Kai Zhang and Tianyu Zheng and Ruoqi Liu and Ge Zhang and Samuel Stevens and Dongfu Jiang and Weiming Ren and Yuxuan Sun and Cong Wei and Botao Yu and Ruibin Yuan and Renliang Sun and Ming Yin and Boyuan Zheng and Zhenzhu Yang and Yibo Liu and Wenhao Huang and Huan Sun and Yu Su and Wenhu Chen},
  year   = {2026},
  eprint = {2311.16502},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/pdf/2311.16502},
  journal = {CVPR 2023 2023}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2311.16502.svg)](https://codeofpaper.com/paper/2311.16502)

Discussion