Related papers: Characterisation of gradient flows on finite state…

We consider a stochastic process which is (a) described by a continuous-time Markov chain on only short time-scales and (b) constrained to conserve a number of hidden quantities on long time-scales. We assume that the transition matrix of…

Consider a Markov chain $\{X_n\}_{n\ge 0}$ with an ergodic probability measure $\pi$. Let $\Psi$ a function on the state space of the chain, with $\alpha$-tails with respect to $\pi$, $\alpha\in (0,2)$. We find sufficient conditions on the…

Constructal Law states that a finite-size flow system that persists in time evolves its configuration so as to provide progressively easier access to the currents that flow through it. Classical Constructal theory derives hierarchical flow…

The time reversal of a completely-positive, nonequilibrium discrete-time quantum Markov evolution is derived via a suitable adjointness relation. Space-time harmonic processes are introduced for the forward and reverse-time transition…

Consider the class of (functions of) strictly stationary Markov chains in which (i) the second moments are finite and (ii) absolute regularity (beta-mixing) is satisfied with exponential mixing rate. For (functions of) Markov chains in that…

In this paper, we abstract a kind of stochastic processes from evolving processes of growing networks, this process is called growing network Markov chains. Thus the existence and the formulas of degree distribution are transformed to the…

In this paper we study the asymptotic behavior of linear processes having as innovations mean zero, square integrable functions of stationary reversible Markov chains. In doing so we shall preserve the generality of coefficients assuming…

We propose certain conditions which are sufficient for the functional law of the iterated logarithm (the Strassen invariance principle) for some general class of non-stationary Markov-Feller chains. This class may be briefly specified by…

In this note we consider a Markov chain formed by a finite system of interacting birth-and-death processes on a finite state space. We study an asymptotic behaviour of the Markov chain as its state space becomes large. In particular, we…

We present a recurrence-transience classification for discrete-time Markov chains on manifolds with negative curvature. Our classification depends only on geometric quantities associated to the increments of the chain, defined via the…

We prove the stability and global convergence of a coupled actor-critic gradient flow for infinite-horizon and entropy-regularised Markov decision processes (MDPs) in continuous state and action space with linear function approximation…

It is known that state-dependent, multi-step Lyapunov bounds lead to greatly simplified verification theorems for stability for large classes of Markov chain models. This is one component of the "fluid model" approach to stability of…

We consider two types of discrete-time Markov chains where the state space is a graded poset and the transitions are taken along the covering relations in the poset. The first type of Markov chain goes only in one direction, either up or…

We study Markov processes conditioned so that their local time must grow slower than a prescribed function. Building upon recent work on Brownian motion with constrained local time in [5] and [33], we study transience and recurrence for a…

The relation between the thermodynamic entropy production and non-Markovian evolutions is matter of current research. Here, we study the behavior of the stochastic entropy production in open quantum systems undergoing unital non-Markovian…

A novel framework for the analysis of observation statistics on time discrete linear evolutions in Banach space is presented. The model differs from traditional models for stochastic processes and, in particular, clearly distinguishes…

Motivated by the notion of resistance distance on graph, we define a new resistance distance between two states on a given finite ergodic Markov chain based on its fundamental matrix. We prove a few equivalent formulations and discuss its…

Using elementary methods, we prove that for a countable Markov chain $P$ of ergodic degree $d > 0$ the rate of convergence towards the stationary distribution is subgeometric of order $n^{-d}$, provided the initial distribution satisfies…

In this paper we establish a diffusion limit for a multivariate continuous time Markov chain whose components are indexed by vertices of a finite graph. The components take values in a common finite set of non-negative integers and evolve…

Markov chains are convenient means of generating realizations of networks with a given (joint or otherwise) degree distribution, since they simply require a procedure for rewiring edges. The major challenge is to find the right number of…