面向类脑神经科学的视觉、听觉与语言基础模型
摘要
认知神经科学 fragmented into specialized models, each tailored to specific experimental paradigms, hence preventing a unified model of cognition in the human brain. Here, we introduce TRIBE v2, a tri-modal (video, audio and language) foundation model capable of predicting human brain activity in a variety of naturalistic and experimental conditions. Leveraging a unified dataset of over 1,000 hours of fMRI across 720 subjects, we demonstrate that our model accurately predicts high-resolution brain responses for novel stimuli, tasks and subjects, superseding traditional linear encoding models, delivering several-fold improvements in accuracy. Critically, TRIBE v2 enables in silico experimentation: tested on seminal visual and neuro-linguistic paradigms, it recovers a variety of results established by decades of empirical research. Finally, by extracting interpretable latent features, TRIBE v2 reveals the fine-grained topography of multisensory integration. These results establish artificial intelligence as a unifying framework for exploring the functional organization of the human brain.
引用
@article{arxiv.2605.04326,
title = {A foundation model of vision, audition, and language for in-silico neuroscience},
author = {Stéphane d'Ascoli and Jérémy Rapin and Yohann Benchetrit and Teon Brooks and Katelyn Begany and Joséphine Raugel and Hubert Banville and Jean-Rémi King},
journal= {arXiv preprint arXiv:2605.04326},
year = {2026}
}