Related papers: Memory equations as reduced Markov processes
We formally extend the notion of Markov order to open quantum processes by accounting for the instruments used to probe the system of interest at different times. Our description recovers the classical Markov order property in the…
Associative memory refers to the ability to relate a memory with an input and targets the restoration of corrupted patterns. It has been intensively studied in classical physical systems, as in neural networks where an attractor dynamics…
We study reinforcement learning with linear function approximation and finite-memory approximations for partially observed Markov decision processes (POMDPs). We first present an algorithm for the value evaluation of finite-memory feedback…
This work leverages recent advances in probabilistic machine learning to discover conservation laws expressed by parametric linear equations. Such equations involve, but are not limited to, ordinary and partial differential,…
Splitting probabilities quantify the likelihood of a given outcome out of competitive events. This key observable of random walk theory, historically introduced as the gambler's ruin problem, is well understood for memoryless (Markovian)…
Experiments, in particular on biological systems, typically probe lower-dimensional observables which are projections of high-dimensional dynamics. In order to infer consistent models capturing the relevant dynamics of the system, it is…
The slow processes of metastable stochastic dynamical systems are difficult to access by direct numerical simulation due the sampling problem. Here, we suggest an approach for modeling the slow parts of Markov processes by approximating the…
Analysis of non-Markovian systems and memory induced phenomena poses an everlasting challenge for physics. As a paradigmatic example we consider a classical Brownian particle of mass $M$ subjected to an external force and exposed to…
In this paper we give explicit examples of power-law correlated stationary Markovian processes y(t) where the stationary pdf shows tails which are gaussian or exponential. These processes are obtained by simply performing a coordinate…
In this paper we present a novel method for learning hierarchical representations of Markov decision processes. Our method works by partitioning the state space into subsets, and defines subtasks for performing transitions between the…
The article discusses a generalization of model of economic growth with constant pace, which takes into account the effects of dynamic memory. Memory means that endogenous or exogenous variable at a given time depends not only on their…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
The aim of this paper is to approximate a finite-state Markov process by another process with fewer states, called herein the approximating process. The approximation problem is formulated using two different methods. The first method,…
Recent benchmarks for memory-augmented reinforcement learning (RL) have introduced partially observable Markov decision process (POMDP) environments in which agents must use historical observations to make decisions. However, these…
We study the problem of deinterleaving a set of finite-memory (Markov) processes over disjoint finite alphabets, which have been randomly interleaved by a finite-memory switch. The deinterleaver has access to a sample of the resulting…
A generalization of the economic model of logistic growth, which takes into account the effects of memory and crises, is suggested. Memory effect means that the economic factors and parameters at any given time depend not only on their…
The classical embeddability problem asks whether a given stochastic matrix $T$, describing transition probabilities of a $d$-level system, can arise from the underlying homogeneous continuous-time Markov process. Here, we investigate the…
Stochastic processes with long memories, known as long memory processes, are ubiquitous in various science and engineering problems. Superposing Markovian stochastic processes generates a non-Markovian long memory process serving as…
Memory effects play a fundamental role in the study of the dynamics of open quantum systems. There exist two conceptually distinct notions of memory discussed for quantum channels in the literature. In quantum information theory quantum…
Using recently proposed measures for non-Markovianity [H. P. Breuer, E. M. Laine, and J. Piilo, Phys. Rev. Lett. {\bf 103}, 210401 (2009)], we study the dynamics of a qubit coupled to a spin environment via an energy-exchange mechanism. We…