Related papers: Realizable monotonicity for continuous-time Markov…

Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models…

We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators…

In this paper we study the problem of model reduction by moment matching for stochastic systems. We characterize the mathematical object which generalizes the notion of moment to stochastic differential equations and we find a class of…

We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…

Non-Markovian quantum processes exhibit different memory effects when measured in different ways; an unambiguous characterization of memory length requires accounting for the sequence of instruments applied to probe the system dynamics.…

We consider long-time behavior of dynamical systems perturbed by a small noise. Under certain conditions, a slow component of such a motion, which is most important for long- time evolution, can be described as a motion on the cone of…

Causal reversibility blends reversibility and causality for concurrent systems. It indicates that an action can be undone provided that all of its consequences have been undone already, thus making it possible to bring the system back to a…

We study the properties of a subclass of stochastic processes called discrete time nonlinear Markov chains with an aggregator, which naturally appear in various topics such as strategic queueing systems, inventory dynamics, opinion…

We characterize synchronization phenomenon in discrete-time, discrete-state random dynamical systems, with random and probabilistic Boolean networks as particular examples. In terms of multiplicative ergodic properties of the induced linear…

This paper studies the set of terminal state covariances that are reachable over a finite time horizon from a given initial state covariance for a linear stochastic system with additive noise. For discrete-time systems, a complete…

We consider Markov chains on partially ordered sets that generalize the success-runs and remaining life chains in reliability theory. We find conditions for recurrence and transience and give simple expressions for the invariant…

This article is a lecture note on the potential theory of (possibly non-reversible) Markov processes and on the connection of this theory with quantitative analysis of the metastability of stochastic processes.

For Markov chains with a partially ordered finite state space we show strong stationary duality under the condition of M\"obius monotonicity of the chain. We show relations of M\"obius monotonicity to other definitions of monotone chains.…

We establish a new Bernstein-type deviation inequality for general (non-reversible) discrete-time Markov chains via an elementary approach. More robust than existing works in the literature, our result only requires the Markov chain to…

We consider two important time scales---the Markov and cryptic orders---that monitor how an observer synchronizes to a finitary stochastic process. We show how to compute these orders exactly and that they are most efficiently calculated…

There is a well-established theory linking certain semi-Markov chains and continuous-time random walks to time-fractional equations and anomalous diffusion. In this work, we go beyond the semi-Markov framework by considering some…

We consider ergodic backward stochastic differential equations in a discrete time setting, where noise is generated by a finite state Markov chain. We show existence and uniqueness of solutions, along with a comparison theorem. To obtain…

We consider generalized definitions of mixing and exactness for random dynamical systems in terms of Markov operator cocycles. We first give six fundamental definitions of mixing for Markov operator cocycles in view of observations of the…

In this paper, we study a notion of local stationarity for discrete time Markov chains which is useful for applications in statistics. In the spirit of some locally stationary processes introduced in the literature, we consider triangular…

Semi-Markov processes generalize Markov processes by adding temporal memory effects as expressed by a semi-Markov kernel. We recall the path weight for a semi-Markov trajectory and the fact that thermodynamic consistency in equilibrium…