Related papers: Generation-by-Generation Dissection of the Respons…
The response of a neuron to synaptic input strongly depends on whether or not it has just emitted a spike. We propose a neuron model that after spike emission exhibits a partial response to residual input charges and study its collective…
We show that the unavoidable increase in neuronal response latency to ongoing stimulation serves as a nonuniform gradual stretching of neuronal circuit delay loops and emerges as an essential mechanism in the formation of various types of…
Recently, different numerical studies of coarsening in disordered systems have shown the existence of a crossover from an initial, transient, power-law domain growth to a slower, presumably logarithmic, growth. However, due to the very slow…
Our brain receives a dynamically changing stream of sensorimotor data. Yet, we perceive a rather organized world, which we segment into and perceive as events. Computational theories of cognitive science on event-predictive cognition…
Scale-free power law structure describes complex networks derived from a wide range of real world processes. The extensive literature focuses almost exclusively on networks with power law exponent strictly larger than 2, which can be…
Generative neural networks learn how to produce highly realistic images from a large, but finite number of examples - or do they simply memorise their training set? To settle this question, Kadkhodaie, Guth, Simoncelli and Mallat (ICLR '24)…
Large Language Models (LLMs) have shown remarkable capabilities across diverse tasks, yet they face inherent limitations such as constrained parametric knowledge and high retraining costs. Retrieval-Augmented Generation (RAG) augments the…
Inhomogeneous temporal processes in natural and social phenomena have been described by bursts that are rapidly occurring events within short time periods alternating with long periods of low activity. In addition to the analysis of…
The impressive performance gains of modern language models currently rely on scaling parameters: larger models store more world knowledge and reason better. Yet compressing all world knowledge into parameters is unnecessary, as only a…
Reasoning models (e.g., DeepSeek-R1) generate long chains of thought to solve harder problems, but they often loop, repeating the same text at low temperatures or with greedy decoding. We study why this happens and what role temperature…
We consider the model of interaction between the immune system and tumor cells including a memory function that reflect the influence of the past states, to simulate the time needed by the latter to develop a chemical and cell mediated…
The storage of large-scale quantum information at finite temperature requires an autonomous and reliable quantum hard drive, also known as a self-correcting quantum memory. It is a long-standing open problem to find a self-correcting…
The Hierarchical Kernel Transformer (HKT) is a multi-scale attention mechanism that processes sequences at L resolution levels via trainable causal downsampling, combining level-specific score matrices through learned convex weights. The…
Large language models (LLMs) such as GPT, Gemini, and Claude often appear adept at solving classic logic puzzles--but how much genuine reasoning underlies their answers? Recent evidence suggests that these models frequently rely on…
We study the linear response to strain in a mean field elastoplastic model for athermal amorphous solids, incorporating the power-law mechanical noise spectrum arising from plastic events. In the "jammed" regime of the model, where the…
Many time series produced by complex systems are empirically found to follow power-law distributions with different exponents $\alpha$. By permuting the independently drawn samples from a power-law distribution, we present non-trivial…
This paper introduces an approach for training o1-like RAG models that retrieve and reason over relevant information step by step before generating the final answer. Conventional RAG methods usually perform a single retrieval step before…
Generative reasoning with large language models (LLMs) often involves long decoding sequences, leading to substantial memory and latency overheads from accumulating key-value (KV) caches. While existing KV compression methods primarily…
A simple cell model consisting of a catalytic reaction network with intermediate complex formation is numerically studied. As nutrients are depleted, the transition from the exponential growth phase to the growth-arrested dormant phase…
Existing approaches for analyzing neural network activations, such as PCA and sparse autoencoders, rely on strong structural assumptions. Generative models offer an alternative: they can uncover structure without such assumptions and act as…