Related papers: Computing expected transition events in reducible …

We derive a simple expression for the probability of trajectories of a master equation. The expression is particularly useful when the number of states is small and permits the calculation of observables that can be defined as functionals…

Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy…

When the initial and transition probabilities of a finite Markov chain in discrete time are not well known, we should perform a sensitivity analysis. This can be done by considering as basic uncertainty models the so-called credal sets that…

This paper introduces a new algorithm for numerically computing equilibrium (i.e. stationary) distributions for Markov chains and Markov jump processes with either a very large finite state space or a countably infinite state space. The…

We study the expected accumulated reward for a discrete-time Markov reward model with absorbing states. The rewards are impulse rewards, where a reward $\rho_{ij}$ is accumulated when transitioning from state $i$ to state $j$. We derive an…

Rare transitions in stochastic processes can often be rigorously described via an underlying large deviation principle. Recent breakthroughs in the classification of reversible stochastic processes as gradient flows have led to a connection…

We study distributions of meeting times for finite symmetric Markov chains. For Markov kernels defined on large state spaces which satisfy certain weak inhomogeneity in return probabilities of points up to large numbers of steps, we obtain…

We test a Markov chain approximation to the segment description (Li, 2007) of chaos (and turbulence) on a tent map, the Minea system, the H\'enon map, and the Lorenz system. For the tent map, we compute the probability transition matrix of…

Transitions in the qualitative behavior of chemical reaction dynamics with a decrease in molecule number have attracted much attention. Here, a method based on a Markov process with a tridiagonal transition matrix is applied to the analysis…

Approximating the stationary probability of a state in a Markov chain through Markov chain Monte Carlo techniques is, in general, inefficient. Standard random walk approaches require $\tilde{O}(\tau/\pi(v))$ operations to approximate the…

For continuous-time ergodic Markov processes, the Kemeny time $\tau_*$ is the characteristic time needed to converge towards the steady state $P_*(x)$ : in real-space, the Kemeny time $\tau_*$ corresponds to the average of the…

Adaptive multilevel splitting algorithms have been introduced rather recently for estimating tail distributions in a fast and efficient way. In particular, they can be used for computing the so-called reactive trajectories corresponding to…

In this paper, we investigate a nonparametric approach to provide a recursive estimator of the transition density of a non-stationary piecewise-deterministic Markov process, from only one observation of the path within a long time. In this…

We consider stochastic discrete event dynamic systems that have time evolution represented with two-dimensional state vectors through a vector equation that is linear in terms of an idempotent semiring. The state transitions are governed by…

We present a numerical method to compute the survival function and the moments of the exit time for a piecewise-deterministic Markov process (PDMP). Our approach is based on the quantization of an underlying discrete-time Markov chain…

Markov automata (MAs) extend labelled transition systems with random delays and probabilistic branching. Action-labelled transitions are instantaneous and yield a distribution over states, whereas timed transitions impose a random delay…

Several phase transitions for excited random walks on the integers are known to be characterized by a certain drift parameter delta. For recurrence/transience the critical threshold is |delta|=1, for ballisticity it is |delta|=2 and for…

The expansion of global production networks has raised many important questions about the interdependence among countries and how future changes in the world economy are likely to affect the countries' positioning in global value chains. We…

We investigate signatures of the excited-state quantum phase transition in the periodic dynamics of the Lipkin-Meshkov-Glick model and the Tavis-Cummings model. In the thermodynamic limit, expectation values of observables in eigenstates of…

We give an example of a transient reversible Markov chain that almost surely has only a finite number of cutpoints. We explain how this is relevant to a conjecture of Diaconis and Freedman and a question of Kaimanovich. We also answer…