Related papers: Memory equations as reduced Markov processes

In this study, a new extension of the Markov Renewal theory is introduced by allowing time to evolve in multiple dimensions. The resulting chains are referred to as multi-time Markov Renewal chains and since this extension is new, the state…

It is common to address the curse of dimensionality in Markov decision processes (MDPs) by exploiting low-rank representations. This motivates much of the recent theoretical study on linear MDPs. However, most approaches require a given…

Generic non-Markovian quantum processes have infinitely long memory, implying an exact description that grows exponentially in complexity with observation time. Here, we present a finite memory ansatz that approximates (or recovers) the…

In this paper, we introduce a class of processes that contains many natural examples. The interesting feature of such type processes lays on its infinite memory that allows it to record a quite ancient history. Then, using the martingale…

Arguably, the largest class of stochastic processes generated by means of a finite memory consists of those that are sequences of observations produced by sequential measurements in a suitable generalized probabilistic theory (GPT). These…

Our work aims at developing reinforcement learning algorithms that do not rely on the Markov assumption. We consider the class of Non-Markov Decision Processes where histories can be abstracted into a finite set of states while preserving…

Among the predictive hidden Markov models that describe a given stochastic process, the {\epsilon}-machine is strongly minimal in that it minimizes every R\'enyi-based memory measure. Quantum models can be smaller still. In contrast with…

Memory is often defined as the mental capacity of retaining information about facts, events, procedures and more generally about any type of previous experience. Memories are remembered as long as they influence our thoughts, feelings, and…

We derive a necessary and sufficient condition for a quantum process to be Markovian which coincides with the classical one in the relevant limit. Our condition unifies all previously known definitions for quantum Markov processes by…

The duration, strength and structure of memory effects are crucial properties of physical evolution. Due to the invasive nature of quantum measurement, such properties must be defined with respect to the probing instruments employed. Here,…

Proteins have been empirically linked to memory. If memory relates to protein structure, then each conformation would_functionally_ code only one bit, making it difficult to explain large memories. Nor is there a simple way to relate memory…

Relational Markov Decision Processes are a useful abstraction for complex reinforcement learning problems and stochastic planning problems. Recent work developed representation schemes and algorithms for planning in such problems using the…

Controlling phase transitions in quantum systems via coupling to reservoirs has been mostly studied for idealized memory-less environments under the so-called Markov approximation. Yet, most quantum materials and experiments in the solid…

This paper is concerned with correlation functions of stochastic systems with memory, a prominent example being a molecule or colloid moving through a complex (e.g., viscoelastic) fluid environment. Analytical investigations of such systems…

Learning arguably involves the discovery and memorization of abstract rules. The aim of this paper is to study associative memory mechanisms. Our model is based on high-dimensional matrices consisting of outer products of embeddings, which…

This note summarizes the optimization formulations used in the study of Markov decision processes. We consider both the discounted and undiscounted processes under the standard and the entropy-regularized settings. For each setting, we…

Methods for the reduction of the complexity of computational problems are presented, as well as their connections to renormalization, scaling, and irreversible statistical mechanics. Several statistically stationary cases are analyzed; for…

This work is a continuation of [Kalikaeva, MPRF, 23(2):225-240]. The object of study is ``Markov-up processes'' on $\mathbb Z_+$ and the moment of downcrossing a certain barrier. The processes considered in this paper differ from Markov…

When is keeping a memory of observations worthwhile? We use hidden Markov models to look at phase transitions that emerge when comparing state estimates in systems with discrete states and noisy observations. We infer the underlying state…

Reinforcement Learning Algorithms are predominantly developed for stationary environments, and the limited literature that considers nonstationary environments often involves specific assumptions about changes that can occur in transition…