Related papers: On moving-average models with feedback

Empirical data on the dynamics of human face-to-face interactions across a variety of social venues have recently revealed a number of context-independent structural and temporal properties of human contact networks. This universality…

Generative Large Language Models (LLMs) have shown promising results in text annotation using zero-shot and few-shot learning. Yet these approaches do not allow the model to retain information from previous annotations, making each response…

Sequence models assign probabilities to variable-length sequences such as natural language texts. The ability of sequence models to capture temporal dependence can be characterized by the temporal scaling of correlation and mutual…

The problem of a random walk in a disordered media is mapped into a model of a random walk with memory. The latter model, as opposed to the former one, does not make reference to a particular realization of the disorder. The equivalence of…

Constant (naive) imputation is still widely used in practice as this is a first easy-to-use technique to deal with missing data. Yet, this simple method could be expected to induce a large bias for prediction purposes, as the imputed input…

The Hopfield associative memory model stores random patterns in synaptic couplings according to Hebb's rule and retrieves them through gradient descent on an energy function. This conventional setting, where neurons are assumed to have…

We study memory based random walk models to understand diffusive motion in crowded heterogeneous environment. The models considered are non-Markovian as the current move of the random walk models is determined by randomly selecting a move…

We investigate attention as the active pursuit of useful information. This contrasts with attention as a mechanism for the attenuation of irrelevant information. We also consider the role of short-term memory, whose use is critical to any…

Longitudinal data are characterized by the dependence between observations coming from the same individual. In a regression perspective, such a dependence can be usefully ascribed to unobserved features (covariates) specific to each…

Human figures have been animated using a variety of geometric models including stick figures, polygonal models, and NURBS-based models with muscles, flexible skin, or clothing. This paper reports on experimental results indicating that a…

Time-invariant linear dynamical system arises in many real-world applications,and its usefulness is widely acknowledged. A practical limitation with this model is that its latent dimension that has a large impact on the model capability…

Models of contagion arise broadly both in the biological and social sciences, with applications ranging from the transmission of infectious diseases to the diffusion of innovations and the spread of cultural fads. In this Letter, we…

Neuron models of associative memory provide a new and prospective technology for reliable date storage and patterns recognition. However, even when the patterns are uncorrelated, the efficiency of most known models of associative memory is…

Sample efficiency is a crucial property of language models with practical implications for training efficiency. In real-world text, information follows a long-tailed distribution. Yet, we expect models to learn and recall frequent and…

Large language models based on the Transformer architecture have demonstrated impressive capabilities to learn in context. However, existing theoretical studies on how this phenomenon arises are limited to the dynamics of a single layer of…

Traditionally, physical models of associative memory assume conditions of equilibrium. Here, we consider a prototypical oscillator model of associative memory and study how active noise sources that drive the system out of equilibrium, as…

The ability to reliably predict the future quality of a wireless channel, as seen by the media access control layer, is a key enabler to improve performance of future industrial networks that do not rely on wires. Knowing in advance how…

Linear Response theory aims to predict how added forcing alters the statistical properties of an unforced system. These kinds of questions have been studied predominantly for autonomous dynamical systems, yet many systems in the physical,…

Modern machine learning models are deployed in diverse, non-stationary environments where they must continually adapt to new tasks and evolving knowledge. Continual fine-tuning and in-context learning are costly and brittle, whereas neural…

Intelligent agents collect and process information from their dynamically evolving neighbourhood to efficiently navigate through it. However, agent-level intelligence does not guarantee that at the level of a collective; a common example is…