MIRB: Mathematical Information Retrieval Benchmark

Haocheng Ju, Bin Dong
5/21/2025
cs.IRcs.CLcs.LG

No official code found. Best implementation: j991222/mirb (3 stars, Apache-2.0, last commit 2025-05-29).

Abstract

Mathematical Information Retrieval (MIR) is the task of retrieving information from mathematical documents and plays a key role in various applications, including theorem search in mathematical libraries, answer retrieval on math forums, and premise selection in automated theorem proving. However, a unified benchmark for evaluating these diverse retrieval tasks has been lacking. In this paper, we introduce MIRB (Mathematical Information Retrieval Benchmark) to assess the MIR capabilities of retrieval models. MIRB includes four tasks: semantic statement retrieval, question-answer retrieval, premise retrieval, and formula retrieval, spanning a total of 12 datasets. We evaluate 13 retrieval models on this benchmark and analyze the challenges inherent to MIR. We hope that MIRB provides a comprehensive framework for evaluating MIR systems and helps advance the development of more effective retrieval models tailored to the mathematical domain.

View on arXivView PDF

Code Implementations(1)

j991222/mirbCommunity100%
Apache-2.0

MIRB: Mathematical Information Retrieval Benchmark

30Shell, PythonMay 16, 20251 years agoApache-2.0
benchmarkretrieval

Similar Papers

SABER-Math: Automated Benchmark for Information Retrieval Evaluation in Mathematics

85% match2026

MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval

81% match2 repos1182026

Discovery and Recognition of Formula Concepts using Machine Learning

80% match1 repo42023

Learning an Effective Premise Retrieval Model for Efficient Mathematical Formalization

79% match1 repo192025

RealMath: A Continuous Benchmark for Evaluating Language Models on Research-Level Mathematics

77% match1 repo202025

Benchmarking Information Retrieval Models on Complex Retrieval Tasks

76% match2025

Cite this paper

@article{ju2025mirb,
  title  = {MIRB: Mathematical Information Retrieval Benchmark},
  author = {Haocheng Ju and Bin Dong},
  year   = {2025},
  eprint = {2505.15585},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2505.15585v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2505.15585.svg)](https://codeofpaper.com/paper/2505.15585)

Discussion