Related papers: A Resolvent Approach to Metastability

We consider the problem of approximating the reachability probabilities in Markov decision processes (MDP) with uncountable (continuous) state and action spaces. While there are algorithms that, for special classes of such MDP, provide a…

Motivated by its connection to the limit behaviour of imprecise Markov chains, we introduce and study the so-called convergence of upper transition operators: the condition that for any function, the orbit resulting from iterated…

The master equation and, more generally, Markov processes are routinely used as models for stochastic processes. They are often justified on the basis of randomization and coarse-graining assumptions. Here instead, we derive n-th order…

Suitable reachability conditions can make two different fixed point semantics of a transition system coincide. For instance, the total and partial expected reward semantics on Markov chains (MCs) coincide whenever the MC at hand is almost…

In this work, we introduce an information-theoretic approach for considering changes in dynamics of finitely dimensional open quantum systems governed by master equations. This experimentally motivated approach arises from considering how…

A generalized method of alternating resolvents was introduced by Boikanyo and Moro{\c s}anu as a way to approximate common zeros of two maximal monotone operators. In this paper we analyse the strong convergence of this algorithm under two…

We consider the problem of non degenerate in energy metastable states forming a series in the framework of reversible finite state space Markov chains. We assume that starting from the state at higher energy the system necessarily visits…

There are some positively divisible non-Markovian processes whose transition matrices satisfy the Chapman-Kolmogorov equation. These processes should also satisfy the Kolmogorov consistency conditions, an essential requirement for a process…

We establish a new Bernstein-type deviation inequality for general (non-reversible) discrete-time Markov chains via an elementary approach. More robust than existing works in the literature, our result only requires the Markov chain to…

Metastability is a spurious mode of operation in digital signals, where an electrical signal fails to settle into a stable state within a specified time, leading to uncertainty and potentially failing downstream hardware. A system that…

We construct and study branching Markov processes on the space of finite configurations of the state space of a given standard process, controlled by a branching kernel and a killing one. In particular, we may start with a superprocess,…

Let $r: S\times S\to \bb R_+$ be the jump rates of an irreducible random walk on a finite set $S$, reversible with respect to some probability measure $m$. For $\alpha >1$, let $g: \bb N\to \bb R_+$ be given by $g(0)=0$, $g(1)=1$, $g(k) =…

We consider symmetric Markov chains on $\Bbb Z^d$ where we do {\bf not} assume that the conductance between two points must be zero if the points are far apart. Under a uniform second moment condition on the conductances, we obtain upper…

We consider the challenge of finding a deterministic policy for a Markov decision process that uniformly (in all states) maximizes one reward subject to a probabilistic constraint over a different reward. Existing solutions do not fully…

We consider processes which are functions of finite-state Markov chains. It is well known that such processes are rarely Markov. However, such processes are often regular in the following sense: the distant past values of the process have…

Given an infinitesimal perturbation of a discrete-time finite Markov chain, we seek the states that are stable despite the perturbation, \textit{i.e.} the states whose weights in the stationary distributions can be bounded away from $0$ as…

We provide a rather explicit formula for the resolvent of a~concatenation of $N$ processes in terms of their exit laws and certain probability measures characterizing the way the processes are concatenated. As an application, we prove an…

Starting from a Markov chain with a finite alphabet, we consider the chain obtained when all but one symbol are undistinguishable for the practitioner. We study necessary and sufficient conditions for this chain to have continuous…

We prove that the limit profile of a sequence of reversible Markov chains exhibiting total variation cutoff is a continuous function, under a computable condition involving the spectrum of the transition matrix and the cutoff window.

We discuss necessary and sufficient conditions for the convergence of disordered asymmetric zero-range process to the critical invariant measures.