Related papers: A note on the state occupancy distribution for Mar…

Studies of fixation dynamics in Markov processes predominantly focus on the mean time to absorption. This may be inadequate if the distribution is broad and skewed. We compute the distribution of fixation times in one-step birth-death…

We investigate the local times of a continuous-time Markov chain on an arbitrary discrete state space. For fixed finite range of the Markov chain, we derive an explicit formula for the joint density of all local times on the range, at any…

This paper deals with a distributed state estimation problem for jointly observable multi-agent systems operated over various time-varying network topologies. The results apply when the system matrix of the system to be observed contains…

To describe and analyze the dynamics of Self-Organized Criticality (SOC) systems, a four-state continuous-time Markov model is proposed in this paper. Different to computer simulation or numeric experimental approaches commonly employed for…

For a network of discrete states with a periodically driven Markovian dynamics, we develop an inference scheme for an external observer who has access to some transitions. Based on waiting-time distributions between these transitions, the…

In this brief note, we find formulas for the distribution and the transition probability matrices of a stochastic process described as a time-reversion in a finite time window of a Markov chain, with cluster observation of the Markov state…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…

Multi-state models are commonly used for intermittent observations of a state over time, but these are generally based on the Markov assumption, that transition rates are independent of the time spent in current and previous states. In a…

This paper proposes a new type of recurrence where we divide the Markov chains into intervals that start when the chain enters into a subset A, then sample another subset B far away from A and end when the chain again return to A. The…

Many biological and medical questions can be modeled using time-to-event data in finite-state Markov chains, with the phase-type distribution describing intervals between events. We solve the inverse problem: given a phase-type…

Extensions of Kemeny's constant, as derived for irreducible finite Markov chains in discrete time, to Markov renewal processes and Markov chains in continuous time are discussed. Three alternative Kemeny's functions and their variants are…

A Markovian dichotomic system driven by a deterministic time-periodic force is analyzed in terms of the statistical properties of the switching events between the states. The consideration of the counting process of the switching events…

Continuous Time Markov Chain (CMTC) is widely used to describe and analyze systems in several knowledge areas. Steady state availability is one important analysis that can be made through Markov chain formalism that allows researchers…

Product-form stationary distributions in Markov chains have been a foundational advance and driving force in our understanding of stochastic systems. In this paper, we introduce a new product-form relationship that we call "graph-based…

In this paper we study the state-feedback stabilization of a discrete-time Markov jump linear system when the observation of the Markov chain of the system, called the Markov state, is time-randomized by another Markov chain. Embedding the…

Markov chains are a convenient means of generating realizations of networks, since they require little more than a procedure for rewiring edges. If a rewiring procedure exists for generating new graphs with specified statistical properties,…

We study an approximation method of stationary characters of a two-dimensional Markov chain via the Stein method. For this purpose, innovative methods are developed to estimate the moments of the Markov chain, as well as the solution to the…

We derive a simple expression for the probability of trajectories of a master equation. The expression is particularly useful when the number of states is small and permits the calculation of observables that can be defined as functionals…

We consider an elementary model for self-organised criticality, the activated random walk on the complete graph. We introduce a discrete time Markov chain as follows. At each time step, we add an active particle at a random vertex and let…