Related papers: The generalized Markov measure as an equilibrium s…
The invariant measure is a fundamental object in the theory of Markov processes. In finite dimensions a Markov process is defined by transition rates of the corresponding stochastic matrix. The Markov tree theorem provides an explicit…
We consider continuous--time Markov kinetics with a finite number of states and a given positive equilibrium distribution P*. For an arbitrary probability distribution $P$ we study the possible right hand sides, dP/dt, of the Kolmogorov…
We prove the existence of equilibrium states for geometric potentials in a class of piecewise weakly convex interval maps. This class includes systems with indifferent fixed points and non-Markov partitions. Under additional hypotheses we…
This paper concentrates on the minimal hitting probability of continuous-time controlled Markov systems (CTCMSs) with countable state and finite admissible action spaces. The existence of an optimal policy is first proved. In particular,…
The entropy production is commonly interpreted as measuring the distance from equilibrium. However, this explanation lacks a rigorous description due to the absence of a natural equilibrium measure. The present analysis formalizes this…
The nonlinear Markov processes are the measure-valued dynamical systems which preserve positivity. They can be represented as the law of large numbers limits of general Markov models of interacting particles. In physics, the kinetic…
We consider the invariant measure of a homogeneous continuous- time Markov process in the quarter-plane. The basic solutions of the global balance equation are the geometric distributions. We first show that the invariant measure can not be…
We show that the joint probability generating function of the stationary measure of a finite state asymmetric exclusion process with open boundaries can be expressed in terms of joint moments of Markov processes called quadratic harnesses.…
For a Markov decision process with countably infinite states, the optimal value may not be achievable in the set of stationary policies. In this paper, we study the existence conditions of an optimal stationary policy in a countable-state…
We study the finiteness of physical measures for skew-product transformations $F$ associated with discrete-time random dynamical systems driven by ergodic Markov chains. We develop a framework, using an independent and identically…
One of the broadest concepts of measurement in quantum theory is the generalized measurement. Another paradigm of measurement--arising naturally in quantum optics, among other fields--is that of continuous-time measurements, which can be…
We consider a large family of discrete and continuous time controlled Markov processes and study an ergodic risk-sensitive minimization problem. Under a blanket stability assumption, we provide a complete analysis to this problem. In…
Random dynamical systems with countably many maps which admit countable Markov partitions on complete metric spaces such that the resulting Markov systems are uniformly continuous and contractive are considered. A non-degeneracy and a…
We consider a piecewise-deterministic Markov process (PDMP) with general conditional distribution of inter-occurrence time, which is called a general PDMP here. Our purpose is to establish the theory of measure-valued generator for general…
In this note we re-visit the fundamental question of the strong law of large numbers and central limit theorem for processes in continuous time with conditional stationary and independent increments. For convenience we refer to them as…
Let $(\xi_n)_{n=0}^\infty$ be a nonhomogeneous Markov chain taking values from finite state-space of $\mathbf{X}=\{1,2,\ldots,b\}$. In this paper, we will study the generalized entropy ergodic theorem with almost-everywhere and…
We present and discuss a general density-matrix description of energy-dissipation and decoherence phenomena in open quantum systems, able to overcome the intrinsic limitations of the conventional Markov approximation. In particular, the…
Many economic models feature monotone Markov dynamics on state spaces that may be noncompact. Establishing existence, uniqueness, and stability of stationary distributions in such settings has required a patchwork of sufficient conditions,…
In a recent paper, Melbourne and Terhesiu [Operator renewal theory and mixing rates for dynamical systems with infinite measure, Invent. Math. 189 (2012), 61-110] obtained results on mixing and mixing rates for a large class of…
We introduce a unified framework to estimate the convergence of Markov chains to equilibrium in Wasserstein distance. The framework can provide convergence bounds with rates ranging from polynomial to exponential, all derived from a…