Related papers: The forget-remember mechanism for 2-state spreadin…
Models for learning probability distributions such as generative models and density estimators behave quite differently from models for learning functions. One example is found in the memorization phenomenon, namely the ultimate convergence…
A model for epidemic spreading on rewiring networks is introduced and analyzed for the case of scale free steady state networks. It is found that contrary to what one would have naively expected, the rewiring process typically tends to…
We consider the generic approach of using an experience memory to help exploration by adapting a restart distribution. That is, given the capacity to reset the state with those corresponding to the agent's past observations, we help…
The iterated learning model simulates the transmission of language from generation to generation in order to explore how the constraints imposed by language transmission facilitate the emergence of language structure. Despite each modelled…
We introduce a modified SIR model with memory for the dynamics of epidemic spreading in a constant population of individuals. Each individual is in one of the states susceptible (${\bf S}$), infected (${\bf I}$) or recovered (${\bf R}$). In…
Several systems can be modeled as sets of interconnected networks or networks with multiple types of connections, here generally called multilayer networks. Spreading processes such as information propagation among users of an online social…
This paper considers continual learning of large-scale pretrained neural machine translation model without accessing the previous training data or introducing model separation. We argue that the widely used regularization-based methods,…
Forgetting is in common in daily life, and 50-80% everyday's forgetting is due to prospective memory failures, which have significant impacts on our life. More seriously, some of these memory lapses can bring fatal consequences such as…
An ability to predict the popularity dynamics of individual items within a complex evolving system has important implications in a wide range of domains. Here we propose a deep learning attention mechanism to model the process through which…
Inspired by the insights in cognitive science with respect to human memory and reasoning mechanism, a novel evolvable LLM-based (Large Language Model) agent framework is proposed as REMEMBERER. By equipping the LLM with a long-term…
In the mammalian brain, newly acquired memories depend on the hippocampus for maintenance and recall, but over time the neocortex takes over these functions, rendering memories hippocampus-independent. The process responsible for this…
The description of a measuring process, such as that which occurs when a quantum point contact (QPC) detector is influenced by a nearby external electron which can take up two possible positions, provides a interesting application of the…
Several mechanisms to focus attention of a neural network on selected parts of its input or memory have been used successfully in deep learning models in recent years. Attention has improved image classification, image captioning, speech…
We introduce a new type of graphical model that we call a "memory factor network" (MFN). We show how to use MFNs to model the structure inherent in many types of data sets. We also introduce an associated message-passing style algorithm…
Many driven systems alternate between bursts of activity and quiescence and can become trapped in an absorbing state, such as complete inactivity in reaction-diffusion processes or extinction in predator-prey dynamics. It is generally…
Event-triggered communication and control provide high control performance in networked control systems without overloading the communication network. However, most approaches require precise mathematical models of the system dynamics,…
Consider a natural language sentence describing a specific step in a food recipe. In such instructions, recognizing actions (such as press, bake, etc.) and the resulting changes in the state of the ingredients (shape molded, custard cooked,…
Numerous recent works target to extend effective context length for language models and various methods, tasks and benchmarks exist to measure model's effective memorization length. However, through thorough investigations, we find…
We extend the recently introduced regularization/Bayesian System Identification procedures to the estimation of time-varying systems. Specifically, we consider an online setting, in which new data become available at given time steps. The…
This paper investigates the second order properties of a stationary process after random sampling. While a short memory process gives always rise to a short memory one, we prove that long-memory can disappear when the sampling law has heavy…