Related papers: Embedded Markov chain approximations in Skorokhod …

Order-preserving couplings are elegant tools for obtaining robust estimates of the time-dependent and stationary distributions of Markov processes that are too complex to be analyzed exactly. The starting point of this paper is to study…

Motivated by applications in Markov chain Monte Carlo, we discuss what it means for one Markov chain to be an approximation to another. Specifically included in that discussion are situations in which a Markov chain with continuous state…

The formal verification of large probabilistic models is important and challenging. Exploiting the concurrency that is often present is one way to address this problem. Here we study a restricted class of asynchronous distributed…

The purpose of this paper is to study the time average behavior of Markov chains with transition probabilities being kernels of completely continuous operators, and therefore to provide a sufficient condition for a class of Markov chains…

We show that the convergence of finite state space Markov chains to stationarity can often be considerably speeded up by alternating every step of the chain with a deterministic move. Under fairly general conditions, we show that not only…

Continuous-time Markov chains describing interacting processes exhibit a state space that grows exponentially in the number of processes. This state-space explosion renders the computation or storage of the time-marginal distribution, which…

We have recently defined a weak Markovian bisimulation equivalence in an integrated-time setting, which reduces sequences of exponentially timed internal actions to individual exponentially timed internal actions having the same average…

Reversible Markov chains play a central role in stochastic modelling and in algorithms such as Markov chain Monte Carlo (MCMC). Motivated by the fundamental importance of reversibility in classical settings, this paper develops a…

Existing studies on the degree correlation of evolving networks typically rely on differential equations and statistical analysis, resulting in only approximate solutions due to inherent randomness. To address this limitation, we propose an…

For Markov chains and Markov processes exhibiting a form of stochastic monotonicity (larger states shift up transition probabilities in terms of stochastic dominance), stability and ergodicity results can be obtained using order-theoretic…

We present an algorithm for learning mixtures of Markov chains and Markov decision processes (MDPs) from short unlabeled trajectories. Specifically, our method handles mixtures of Markov chains with optional control input by going through a…

In this brief paper we find computable exponential convergence rates for a large class of stochastically ordered Markov processes. We extend the result of Lund, Meyn, and Tweedie (1996), who found exponential convergence rates for…

We show how to map the states of an ergodic Markov chain to Euclidean space so that the squared distance between states is the expected commuting time. We find a minimax characterization of commuting times, and from this we get monotonicity…

The use of higher-order stochastic processes such as nonlinear Markov chains or vertex-reinforced random walks is significantly growing in recent years as they are much better at modeling high dimensional data and nonlinear dynamics in…

We consider the computational task of sampling a bit string $x$ from a distribution $\pi(x)=|\langle x|\psi\rangle|^2$, where $\psi$ is the unique ground state of a local Hamiltonian $H$. Our main result describes a direct link between the…

Hybrid systems, and Piecewise Deterministic Markov Processes in particular, are widely used to model and numerically study systems exhibiting multiple time scales in biochemical reaction kinetics and related areas. In this paper an almost…

We present an approach for testing for the existence of continuous generators of discrete stochastic transition matrices. Typically, the known approaches to ascertain the existence of continuous Markov processes are based in the assumption…

Sewer pipe systems are essential for social and economic welfare. Managing these systems requires robust predictive models for degradation behaviour. This study focuses on probability-based approaches, particularly Markov chains, for their…

Inter-channel mis-synchronisation can be a limiting factor to the time resolution of high performance timing detectors with multiple readout channels and independent electronics units. In these systems, time calibration methods employed…

Stochastic convergence of discrete time Markov processes has been analysed based on a dual Lyapunov approach. Using some existing results on ergodic theory of Markov processes, it has been shown that existence of a properly subinvariant…