Related papers: Generation-by-Generation Dissection of the Respons…

The generalized Langevin equation (GLE) is a universal model for particle velocity in a viscoelastic medium. In this paper, we consider the GLE family with fractional memory kernels. We show that, in the critical regime where the memory…

Working memory -- the ability to store and recall precise temporal patterns of neural activity -- remains an open challenge for spiking neural networks (SNNs). We propose a recurrent SNN of $N$ neurons in which each synapse is equipped with…

We discover that large language models exhibit \emph{spectral phase transitions} in their hidden activation spaces when engaging in reasoning versus factual recall. Through systematic spectral analysis across \textbf{11 models} spanning…

This thesis addresses whether it is possible to build a robust memory device for quantum information. A three-dimensional gapped lattice spin model is found which demonstrates for the first time that a reliable quantum memory at finite…

We present the first exact analysis of some of the temporal properties of multivariate self-excited Hawkes conditional Poisson processes, which constitute powerful representations of a large variety of systems with bursty events, for which…

We consider a reaction-diffusion process with retardation. The particles, immersed in traps initially, remain inactive until another particle is annihilated spontaneously with a rate $\lambda$ at a certain point $\vec x$. In that case the…

We study time series concerning rare events. The occurrence of a rare event is depicted as a jump of constant intensity always occurring in the same direction, thereby generating an asymmetric diffusion process. We consider the case where…

An autoregressive model with a power-law type memory kernel is studied as a stochastic process that exhibits a self-affine-fractal-like behavior for a small time scale. We find numerically that the root-mean-square displacement for the time…

To improve the reasoning capabilities of large language models, test-time compute is typically scaled by generating intermediate tokens before the final answer. However, this couples reasoning to autoregressive generation and thereby…

The prevailing assumption of an exponential decay in large language model (LLM) reliability with sequence length, predicated on independent per-token error probabilities, posits an inherent limitation for long autoregressive outputs. Our…

Generative retrieval reformulates retrieval as an autoregressive generation task, where large language models (LLMs) generate target documents directly from a query. As a novel paradigm, the mechanisms that underpin its performance and…

Retrieval-Augmented Generation (RAG) grounds large language model outputs in external evidence, but remains challenged on multi-hop question answering that requires long reasoning. Recent works scale RAG at inference time along two…

Emergence is a fascinating property of large language models and neural networks more broadly: as models scale and train for longer, they sometimes develop new abilities in sudden ways. Despite initial studies, we still lack a comprehensive…

We prove regenerative properties for the linear Hawkes process under minimal assumptions on the transfer function, which may have unbounded support. These results are applicable to sliding window statistical estimators. We exploit…

Transformer LLMs have been shown to exhibit strong reasoning ability that scales with inference-time compute, most prominently through token-space "thinking" chains of thought. A growing line of work pushes extra computation into the…

Task replication has recently been advocated as a practical solution to reduce latencies in parallel systems. In addition to several convincing empirical studies, some others provide analytical results, yet under some strong assumptions…

Transformer models struggle with long-context inference due to their quadratic time and linear memory complexity. Recurrent Memory Transformers (RMTs) offer a solution by reducing the asymptotic cost to linear time and constant memory…

This paper introduces kernel continual learning, a simple but effective variant of continual learning that leverages the non-parametric nature of kernel methods to tackle catastrophic forgetting. We deploy an episodic memory unit that…

In this article, we investigate the ergodic behaviour of a multidimensional age-dependent branching process with a singular jump kernel, motivated by studying the phenomenon of telomere shortening in cell populations. Our model tracks…

Current deep neural networks can achieve remarkable performance on a single task. However, when the deep neural network is continually trained on a sequence of tasks, it seems to gradually forget the previous learned knowledge. This…