Related papers: Memory equations as reduced Markov processes

The emergence of memory is a hallmark feature of non-Markovian dynamics. However, the type of memory -- classical or quantum -- required to realize certain dynamics remains unknown. We study the quantum homogenizer as a minimal model of…

We consider large-scale Markov decision processes (MDPs) with a risk measure of variability in cost, under the risk-aware MDPs paradigm. Previous studies showed that risk-aware MDPs, based on a minimax approach to handling risk, can be…

Memory behavior modeling is a core issue in cognitive psychology and education. Classical psychological theories typically use memory equations to describe memory behavior, which exhibits insufficient accuracy and controversy, while…

Under a markovian evolutionary process, the expected number of substitutions per site (also called branch length) that have occurred when a sequence has evolved from another according to a transition matrix $P$ can be approximated by…

We develop a quadratic regularization approach for the solution of high-dimensional multistage stochastic optimization problems characterized by a potentially large number of time periods/stages (e.g. hundreds), a high-dimensional resource…

Non-Markovian effects in quantum evolution appear when the system is strongly coupled to the environment and interacts with it for long periods of time. To include memory effects in the master equations, one usually incorporates time-local…

A rescaled Markov chain converges uniformly in probability to the solution of an ordinary differential equation, under carefully specified assumptions. The presentation is much simpler than those in the outside literature. The result may be…

Stochastic processes with random reinforced relocations have been introduced in the physics literature to model animal foraging behaviour. Such a process evolves as a Markov process, except at random relocation times, when it chooses a time…

Pathways of diffusion observed in real-world systems often require stochastic processes going beyond first-order Markov models, as implicitly assumed in network theory. In this work, we focus on second-order Markov models, and derive an…

We consider large-scale Markov decision processes (MDPs) with parameter uncertainty, under the robust MDP paradigm. Previous studies showed that robust MDPs, based on a minimax approach to handle uncertainty, can be solved using dynamic…

We analyse large deviations of time-averaged quantities in stochastic processes with long-range memory, where the dynamics at time t depends itself on the value q_t of the time-averaged quantity. First we consider the elephant random walk…

Friction is a phenomenon that manifests across all spatial and temporal scales, from the molecular to the macroscopic scale. It describes the dissipation of energy from the motion of particles or abstract reaction coordinates and arises in…

We consider a living organism as an observer of the evolution of its environment recording sensory information about the state space X of the environment in real time. Sensory information is sampled and then processed on two levels. On the…

Order-preserving couplings are elegant tools for obtaining robust estimates of the time-dependent and stationary distributions of Markov processes that are too complex to be analyzed exactly. The starting point of this paper is to study…

Quantum memory effects are essential in understanding and controlling open quantum systems, yet distinguishing them from classical memory remains challenging. We introduce a convex geometric framework to analyze quantum memory propagating…

A Markov decision process can be parameterized by a transition kernel and a reward function. Both play essential roles in the study of reinforcement learning as evidenced by their presence in the Bellman equations. In our inquiry of various…

Many chemical reactions and molecular processes occur on timescales that are significantly longer than those accessible by direct simulation. One successful approach to estimating dynamical statistics for such processes is to use many short…

Analyzing large volumes of high-dimensional data requires dimensionality reduction: finding meaningful low-dimensional structures hidden in their high-dimensional observations. Such practice is needed in atomistic simulations of complex…

Non-linear Hawkes processes with memory kernels given by the sum of Erlang kernels are considered. It is shown that their stability properties can be studied in terms of an associated class of piecewise deterministic Markov processes,…

Classical models of memory in psychology and neuroscience rely on similarity-based retrieval of stored patterns, where similarity is a function of retrieval cues and the stored patterns. While parsimonious, these models do not allow…