Related papers: Multimodal neural networks better explain multivox…
A key question in neuroscience is at which level functional meaning emerges from biophysical phenomena. In most vertebrate systems, precise functions are assigned at the level of neural populations, while single-neurons are deemed…
This paper examines the memory capacity of generalized neural networks. Hopfield networks trained with a variety of learning techniques are investigated for their capacity both for binary and non-binary alphabets. It is shown that the…
Synaptic plasticity and neuron cross-talk are some of the important key mechanisms underlying formation of dynamic clusters of active neurons. The essence of this study is to model and decipher the mechanism of emergence of a task-specific…
To interpret deep networks, one main approach is to associate neurons with human-understandable concepts. However, existing methods often ignore the inherent relationships of different concepts (e.g., dog and cat both belong to animals),…
Brain metabolism is controlled by complex regulation mechanisms. As part of their nature many complex systems show scaling behavior in their timeseries data. Corresponding scaling exponents can sometimes be used to characterize these…
Deciphering brain function through non-invasive recordings requires synthesizing complementary high-frequency electromagnetic (EEG/MEG) and low-frequency metabolic (fMRI) signals. However, despite their shared neural origins, extreme…
This paper describes a framework for modeling the interface between perception and memory on the algorithmic level of analysis. It is consistent with phenomena associated with many different brain regions. These include view-dependence (and…
Topological data analyses are rapidly turning into key tools for quantifying large volumes of neurobiological data, e.g., for organizing the spiking outputs of large neuronal ensembles and thus gaining insights into the information produced…
The task of multimodal learning has seen a growing interest recently as it allows for training neural architectures based on different modalities such as vision, text, and audio. One challenge in training such models is that they need to…
The interaction between syntax (formal language) and its semantics (meanings of language) is one which has been well studied in categorical logic. The results of this particular study are employed to understand how the brain is able to…
The neural networks of the brain are capable of learning statistical input regularities on the basis of synaptic learning, functional integration into increasingly larger, interconnected neural assemblies, and self organization. This self…
Sequences of neuronal activation have long been implicated in a variety of brain functions. In particular, these sequences have been tied to memory formation and spatial navigation in the hippocampus, a region of mammalian brains.…
Multimodal AI models have achieved impressive performance in tasks that require integrating information from multiple modalities, such as vision and language. However, their "black-box" nature poses a major barrier to deployment in…
Multimodal processing has attracted much attention lately especially with the success of pre-training. However, the exploration has mainly focused on vision-language pre-training, as introducing more modalities can greatly complicate model…
Mechanistic interpretability aims to understand how models store representations by breaking down neural networks into interpretable units. However, the occurrence of polysemantic neurons, or neurons that respond to multiple unrelated…
Cognitive neuroscience is fragmented into specialized models, each tailored to specific experimental paradigms, hence preventing a unified model of cognition in the human brain. Here, we introduce TRIBE v2, a tri-modal (video, audio and…
Concept vectors aim to enhance model interpretability by linking internal representations with human-understandable semantics, but their utility is often limited by noisy and inconsistent activations. In this work, we uncover a clear…
Large pre-trained vision-language models, such as CLIP, have demonstrated state-of-the-art performance across a wide range of image classification tasks, without requiring retraining. Few-shot CLIP is competitive with existing specialized…
Recent advances in large language models (LLMs) have led to the development of multimodal LLMs (MLLMs) in the fields of natural language processing (NLP) and computer vision. Although these models allow for integrated visual and language…
Multi-hop machine reading comprehension is a challenging task in natural language processing as it requires more reasoning ability across multiple documents. Spectral models based on graph convolutional networks have shown good inferring…