The Multimodal Universe: Enabling Large-Scale Machine Learning with 100 TB of Astronomical Scientific Data

The Multimodal Universe Collaboration. Eirini Angeloudi, Jeroen Audenaert, Micah Bowles, B. M. Boyd, David Chemaly +24 more
2/3/2026

No official code found. Best implementation: MultimodalUniverse/MultimodalUniverse (502 stars, MIT, last commit 2026-04-08).

Abstract

We present the MULTIMODAL UNIVERSE, a large-scale multimodal dataset of scientific astronomical data, compiled specifically to facilitate machine learning research. Overall, the MULTIMODAL UNIVERSE contains hundreds of millions of astronomical observations, constituting 100\,TB of multi-channel and hyper-spectral images, spectra, multivariate time series, as well as a wide variety of associated scientific measurements and"metadata". In addition, we include a range of benchmark tasks representative of standard practices for machine learning methods in astrophysics. This massive dataset will enable the development of large multi-modal models specifically targeted towards scientific applications. All codes used to compile the MULTIMODAL UNIVERSE and a description of how to access the data is available at https://github.com/MultimodalUniverse/MultimodalUniverse

View on arXivView PDF

Code Implementations(1)

MultimodalUniverse/MultimodalUniverseCommunity73%
MIT

Large-Scale Multimodal Dataset of Astronomical Data

50268Shell, PythonOct 23, 20236 months agoMIT

Similar Papers

AstroM$^3$: A self-supervised multimodal model for astronomy

82% match2024

SM-Net: Learning a Continuous Spectral Manifold from Multiple Stellar Libraries

82% match1 repo2026

Scalable Cosmic AI Inference using Cloud Serverless Computing

81% match1 repo42025

A Foundation Model for the Solar Dynamics Observatory

80% match2024

XLRS-Bench: Could Your Multimodal LLMs Understand Extremely Large Ultra-High-Resolution Remote Sensing Imagery?

80% match2026

Unsupervised Machine Learning for the Classification of Astrophysical X-ray Sources

79% match2 repos102024

Cite this paper

@article{angeloudi2026multimodal,
  title  = {The Multimodal Universe: Enabling Large-Scale Machine Learning with 100 TB of Astronomical Scientific Data},
  author = {The Multimodal Universe Collaboration. Eirini Angeloudi and Jeroen Audenaert and Micah Bowles and B. M. Boyd and David Chemaly and B. Cherinka and Ioana Ciucua and Miles Cranmer and Aaron Do and M. Grayling and Erin E Hayes and Tom Hehir and Shirley Ho and M. Huertas-Company and Kartheik G. Iyer and Maja Jabłońska and François Lanusse and Henry W. Leung and K. Mandel and J. R. Mart'inez-Galarza and Peter Melchior and Lucas Meyer and L. Parker and Helen Qu and Jeff Shen and Michael J. Smith and Connor Stone and Mike Walmsley and John F. Wu},
  year   = {2026},
  eprint = {2412.02527},
  archivePrefix = {arXiv},
  doi    = {10.52202/079017-1845},
  url    = {https://arxiv.org/abs/2412.02527},
  journal = {NEURIPS 2024 2024}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2412.02527.svg)](https://codeofpaper.com/paper/2412.02527)

Discussion