Related papers: Constructing Strong Markov Processes
We are considering typed hierarchies of total, continuous functionals using complete, separable metric spaces at the base types. We pay special attention to the so called Urysohn space constructed by P. Urysohn. One of the properties of the…
Suppose $ E$ is a space with a null-recurrent Markov kernel $ P$. Furthermore, suppose there are infinite particles with variable weights on $ E$ performing a random walk following $ P$. Let $ X_{t}$ be a weighted functional of the position…
The practically important classes of equal-input and of monotone Markov matrices are revisited, with special focus on embeddability, infinite divisibility, and mutual relations. Several uniqueness results for the classic Markov embedding…
We discuss a canonical structure that provides a unifying description of dynamical large deviations for irreversible finite state Markov chains (continuous time), Onsager theory, and Macroscopic Fluctuation Theory. For Markov chains, this…
The master equation and, more generally, Markov processes are routinely used as models for stochastic processes. They are often justified on the basis of randomization and coarse-graining assumptions. Here instead, we derive n-th order…
Nonlinear Markov chains with finite state space have been introduced in Kolokoltsov (2010). The characteristic property of these processes is that the transition probabilities do not only depend on the state, but also on the distribution of…
We present metrics for measuring the similarity of states in a finite Markov decision process (MDP). The formulation of our metrics is based on the notion of bisimulation for MDPs, with an aim towards solving discounted infinite horizon…
Markovianity of the quantum open system processes is a topic of the considerable current interest. Typically, invertibility is assumed to be non-essential for Markovianity of the open-quantum-system dynamical maps. Nevertheless, in this…
Mean-field models are often used to approximate Markov processes with large state-spaces. One-step processes, also known as birth-death processes, are an important class of such processes and are processes with state space…
Recently, a class of stochastic processes known as piecewise deterministic Markov processes has been used to define continuous-time Markov chain Monte Carlo algorithms with a number of attractive properties, including compatibility with…
Markov decision processes (MDPs) are a fundamental model in sequential decision making. Robust MDPs (RMDPs) extend this framework by allowing uncertainty in transition probabilities and optimizing against the worst-case realization of that…
Reinforcement learning algorithms are typically designed for generic Markov Decision Processes (MDPs), where any state-action pair can lead to an arbitrary transition distribution. In many practical systems, however, only a subset of the…
Generators of Markov processes on a countable state space can be represented as finite or infinite matrices. One key property is that the off-diagonal entries corresponding to jump rates of the Markov process are non-negative. Here we…
Reinforcement learning in non-stationary environments is challenging due to abrupt and unpredictable changes in dynamics, often causing traditional algorithms to fail to converge. However, in many real-world cases, non-stationarity has some…
We present a general approach for computing the dynamic partition function of a continuous-time Markov process. The Ruelle topological pressure is identified with the large deviation function of a physical observable. We construct for the…
Markov process is widely applied in almost all aspects of literature, especially important for understanding non-equilibrium processes. We introduce a decomposition to general Markov process in this paper. This decomposition decomposes the…
Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task whose outcome's quality depends on both the amount and the diversity of the sampled regions of the state-action space. Yet, many MDPs are…
We introduce a class of so called Markovian marginals, which gives a natural framework for constructing solutions to the quantum marginal problem. We consider a set of marginals that possess a certain internal quantum Markov chain…
The paper is devoted to the integral functionals $\int_0^\infty f(X_t)\,{\mathrm{d}t}$ of Markov processes in $\X$ in the case $d\ge 3$. It is established that such functionals can be presented as the integrals $\int_{\X} f(y) \G(x,…
In typical single-molecule force spectroscopy experiments the mechanical unfolding of molecular complexes or biomolecules is studied applying a force ramp to one end of the system while the other end is kept fixed in space. The…