A foundation model of vision, audition, and language for in-silico neuroscience

Stéphane d'Ascoli, Jérémy Rapin, Yohann Benchetrit, Teon Brooks, Katelyn Begany +3 more
5/5/2026
q-bio.NCcs.LG

No official code found. Best implementation: piandpower/suno-feel-lab (0 stars, Apache-2.0, last commit 2026-06-10).

Abstract

Cognitive neuroscience is fragmented into specialized models, each tailored to specific experimental paradigms, hence preventing a unified model of cognition in the human brain. Here, we introduce TRIBE v2, a tri-modal (video, audio and language) foundation model capable of predicting human brain activity in a variety of naturalistic and experimental conditions. Leveraging a unified dataset of over 1,000 hours of fMRI across 720 subjects, we demonstrate that our model accurately predicts high-resolution brain responses for novel stimuli, tasks and subjects, superseding traditional linear encoding models, delivering several-fold improvements in accuracy. Critically, TRIBE v2 enables in silico experimentation: tested on seminal visual and neuro-linguistic paradigms, it recovers a variety of results established by decades of empirical research. Finally, by extracting interpretable latent features, TRIBE v2 reveals the fine-grained topography of multisensory integration. These results establish artificial intelligence as a unifying framework for exploring the functional organization of the human brain.

View on arXivView PDF

Code Implementations(1)

piandpower/suno-feel-labCommunity25%
Apache-2.0

Applying "A foundation model of vision, audition, and language for in-silico neuroscience" to generated content.

00CSS, HTMLJun 10, 20264 months agoApache-2.0

Similar Papers

OmniMouse: Scaling properties of multi-modal, multi-task Brain Models on 150B Neural Tokens

88% match1 repo202026

Uncovering Brain-Like Hierarchical Patterns in Vision-Language Models through fMRI-Based Neural Encoding

86% match2025

RABBiT: Rapidly adaptive BOLD foundation model via brain-tuning for accurate zero-shot and few-shot prediction of speech-elicited responses in the brain

85% match2026

Brain-like Functional Organization within Large Language Models

85% match2024

Deep Neural Networks and Brain Alignment: Brain Encoding and Decoding (Survey)

85% match1 repo512023

TRIBE: TRImodal Brain Encoder for whole-brain fMRI response prediction

84% match1 repo3152025

Cite this paper

@article{dascoli2026foundation,
  title  = {A foundation model of vision, audition, and language for in-silico neuroscience},
  author = {Stéphane d'Ascoli and Jérémy Rapin and Yohann Benchetrit and Teon Brooks and Katelyn Begany and Joséphine Raugel and Hubert Banville and Jean-Rémi King},
  year   = {2026},
  eprint = {2605.04326},
  archivePrefix = {arXiv},
  url    = {http://arxiv.org/abs/2605.04326v1}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/2605.04326.svg)](https://codeofpaper.com/paper/2605.04326)

Discussion