Related papers: Conditional Markov Chains Part II: Consistency and…
We consider two types of discrete-time Markov chains where the state space is a graded poset and the transitions are taken along the covering relations in the poset. The first type of Markov chain goes only in one direction, either up or…
We introduce a Markov chain model of concurrent quantum programs. This model is a quantum generalization of Hart, Sharir and Pnueli's probabilistic concurrent programs. Some characterizations of the reachable space, uniformly repeatedly…
In a previous paper we determined one dimensional distributions of a stationary field with linear regressions and quadratic conditional variances under a linear constraint on the coefficients of the quadratic expression. In this paper we…
A rigorous and largely self-contained account of (a) the bread-and-butter concepts and techniques in Markov chain theory and (b) the long-term behaviour of chains. As much as possible, the treatment is probabilistic instead of analytical (I…
We consider the analysis of sets of categorical sequences consisting of piecewise homogeneous Markov segments. The sequences are assumed to be governed by a common underlying process with segments occurring in the same order for each…
Convergence rates of Markov chains have been widely studied in recent years. In particular, quantitative bounds on convergence rates have been studied in various forms by Meyn and Tweedie [Ann. Appl. Probab. 4 (1994) 981-1101], Rosenthal…
A new definition of continuous-time equilibrium controls is introduced. As opposed to the standard definition, which involves a derivative-type operation, the new definition parallels how a discrete-time equilibrium is defined, and allows…
We show that if the density of the absolutely continuous part of a copula is bounded away from zero on a set of Lebesgue measure 1, then that copula generates \textquotedblleft lower $\psi$-mixing\textquotedblright\ stationary Markov…
We introduce a Markov product structure for multivariate tail dependence functions, building upon the well-known Markov product for copulas. We investigate algebraic and monotonicity properties of this new product as well as its role in…
In this paper we prove a sharp quantitative version of the Kendall's Theorem. The Kendal Theorem states that under some mild conditions imposed on a probability distribution on positive integers (i.e. probabilistic sequence) one can prove…
Many problems of practical interest rely on Continuous-time Markov chains~(CTMCs) defined over combinatorial state spaces, rendering the computation of transition probabilities, and hence probabilistic inference, difficult or impossible…
We consider Markov processes, which describe e.g. queueing network processes, in a random environment which influences the network by determining random breakdown of nodes, and the necessity of repair thereafter. Starting from an explicit…
Two approaches to studying the correlation functions of the binary Markov sequences are considered. The first of them is based on the study of probability of occurring different ''words'' in the sequence. The other one uses recurrence…
We address the problem of community detection in networks by introducing a general definition of Markov stability, based on the difference between the probability fluxes of a Markov chain on the network at different time scales. The…
We study the problem of characterizing the expected hitting times for a robust generalization of continuous-time Markov chains. This generalization is based on the theory of imprecise probabilities, and the models with which we work…
Recursive stochastic algorithms have gained significant attention in the recent past due to data driven applications. Examples include stochastic gradient descent for solving large-scale optimization problems and empirical dynamic…
In this note, a general approach to the study of non-stationary Markov chains with catastrophes and the corresponding queuing models is considered, as well as to obtain estimates of the limiting regime itself. As an illustration, an example…
Continuous-time Markov chains are used to model stochastic systems where transitions can occur at irregular times, e.g., birth-death processes, chemical reaction networks, population dynamics, and gene regulatory networks. We develop a…
Motivated by applications in Markov chain Monte Carlo, we discuss what it means for one Markov chain to be an approximation to another. Specifically included in that discussion are situations in which a Markov chain with continuous state…
If $(C_n)$ is a Markov chain on a discrete state space ${\mathcal{S}}$, a Markov chain $(C_n,M_n)$ on the product space ${\mathcal{S}}\times{\mathcal{S}}$, the cat and mouse Markov chain, is constructed. The first coordinate of this Markov…