Related papers: Memory equations as reduced Markov processes
In recent years it has become clear that the brain maintains a temporal memory of recent events stretching far into the past. This paper presents a neurally-inspired algorithm to use a scale-invariant temporal representation of the past to…
We study the structure of quantum Markov Processes from the point of view of product systems and their representations.
Time delays increase the effective dimensionality of reservoirs, thus suggesting that time delays in reservoirs can enhance their performance, particularly their memory and prediction abilities. We find new closed-form expressions for…
The orientational memory of particles can serve as an effective measure of diffusivity, spreading, and search efficiency in complex stochastic processes. We develop a theoretical framework to describe the decay of directional correlations…
We consider a discrete time semi-Markov process where the characteristics defining the process depend on a small perturbation parameter. It is assumed that the state space consists of one finite communicating class of states and, in…
The underdamped, non-linear, generalized Langevin equation is widely used to model coarse-grained dynamics of soft and biological materials. By means of a projection operator formalism, we show under which approximations this equation can…
Markov decision problems (MDPs) provide the foundations for a number of problems of interest to AI researchers studying automated planning and reinforcement learning. In this paper, we summarize results regarding the complexity of solving…
In a quantum (inhomogeneous) Markov process $\rho_1:=\Gamma_1(\rho)$, $\rho_2:=\Gamma_1(\rho_1)$, ..., where $\Gamma_i$ are CPTP maps and $\rho$ is the initial state, the the state of the system is either oscillatory or convergent to a…
In the paper the memory effect in the system consisting from a trajectory of process and an environment is considered. The environment is presented by scalar potential and noise. The evolution of system is interpreted as process of the…
This paper presents a technique for reduced-order Markov modeling for compact representation of time-series data. In this work, symbolic dynamics-based tools have been used to infer an approximate generative Markov model. The time-series…
The stochastic properties of a Langevin-type Markov process can be extracted from a given time series by a Markov analysis. Also processes that obey a stochastically forced second order differential equation can be analyzed this way by…
We treat the class of universal Markov processes on the d-dimensional Euklidean space which do not depend on random. For these, as well as for several subclasses, we prove criteria whether a function f, defined on the positive half-line,…
The basic features of the dynamics of open quantum systems, such as the dissipation of energy, the decay of coherences, the relaxation to an equilibrium or non-equilibrium stationary state, and the transport of excitations in complex…
Markov decision processes continue to gain in popularity for modeling a wide range of applications ranging from analysis of supply chains and queuing networks to cognitive science and control of autonomous vehicles. Nonetheless, they tend…
We present an investigation of stochastic evolution in which a family of evolution equations in $L^1$ are driven by continuous-time Markov processes. These are examples of so-called piecewise deterministic Markov processes (PDMP's) on the…
Markov matrices have an important role in the filed of stochastic processes. In this paper, we will show and prove a series of conclusions on Markov matrices and transformations rather than pay attention to stochastic processes although…
Reformulating the history matching problem from a least-square mathematical optimization problem into a Markov Decision Process introduces a method in which reinforcement learning can be utilized to solve the problem. This method provides a…
We consider the evolution of correlation functions in a non-Markov version of the contact model in the continuum. The memory effects are introduced by assuming the fractional evolution equation for the statistical dynamics. This leads to a…
In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…
In Partially Observable Markov Decision Processes, integrating an agent's history into memory poses a significant challenge for decision-making. Traditional imitation learning, relying on observation-action pairs for expert demonstrations,…