Related papers: The optimal sink and the best source in a Markov c…

We consider the Markov chain approximations for singular stable-like processes. First we obtain properties of some Markov chains. Then we construct the approximating Markov chains and give a necessary condition for weak convergence of these…

In this work, we consider a finite-state inhomogeneous-time Markov chain whose probabilities of transition from one state to another tend to decrease over time. This can be seen as a cooling of the dynamics of an underlying Markov chain. We…

Locally Markov walks are natural generalizations of classical Markov chains, where instead of a particle moving independently of the past, it decides where to move next depending on the last action performed at the current location. We…

We prove that transport in the phase space of the "most strongly chaotic" dynamical systems has three different stages. Consider a finite Markov partition (coarse graining) $\xi$ of the phase space of such a system. In the first short times…

Nonlinear Markov chains with finite state space have been introduced in Kolokoltsov (2010). The characteristic property of these processes is that the transition probabilities do not only depend on the state, but also on the distribution of…

We revisit the source coding problem for a Markov chain under the assumption that the transmission times and how fast the Markov chain transitions its state happen at the same time-scale. Specifically, we assume that the transmission of…

We obtain universal estimates on the convergence to equilibrium and the times of coupling for continuous time irreducible reversible finite-state Markov chains, both in the total variation and in the L^2 norms. The estimates in total…

We study Markov processes where the "time" parameter is replaced by paths in a directed graph from an initial vertex to a terminal one. Along each directed path the process is Markov and has the same distribution as the one along any other…

For non-equilibrium systems described by finite Markov processes, we consider the number of times that a system traverses a cyclic sequence of states (a cycle). The joint distribution of the number of forward and backward instances of any…

The class of nonlinear Markov processes is characterized by the dependence of the current state of the process on its current distribution in addition to the dependence on the previous state. Due to this feature, these processes are…

When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This can be done by considering as basic uncertainty models the so-called credal sets that…

We consider Markov chains on partially ordered sets that generalize the success-runs and remaining life chains in reliability theory. We find conditions for recurrence and transience and give simple expressions for the invariant…

A discrete time branching process where the offspring distribution is generation-dependent, and the number of reproductive individuals is controlled by a random mechanism is considered. This model is a Markov chain but, in general, the…

Standard Markovian optimal stopping problems are consistent in the sense that the first entrance time into the stopping set is optimal for each initial state of the process. Clearly, the usual concept of optimality cannot in a…

We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators…

Suppose X and Y are two independent irreducible Markov chains on n states. We consider the intersection time, which is the first time their trajectories intersect. We show for reversible and lazy chains that the total variation mixing time…

We give an example of a transient reversible Markov chain that almost surely has only a finite number of cutpoints. We explain how this is relevant to a conjecture of Diaconis and Freedman and a question of Kaimanovich. We also answer…

The main subject of the study in this paper is the simultaneous renewal time for two time-inhomogeneous Markov chains which start with arbitrary initial distributions. By a simultaneous renewal we mean the first time of joint hitting the…

We classify the possible behaviors of a class of one-dimensional stochastic recurrent growth models. In our main result, we obtain nearly optimal bounds for the tail of hitting times of some compact sets. If the process is an aperiodic…

In probability theory, equalities are much less than inequalities. In this paper, we find a series of equalities which characterize the symmetry of the forming times of a family of similar cycles for discrete-time and continuous-time Markov…