Related papers: A note on the state occupancy distribution for Mar…
An (upward) skip-free Markov chain with the set of nonnegative integers as state space is a chain for which upward jumps may be only of unit size; there is no restriction on downward jumps. In a 1987 paper, Brown and Shao determined, for an…
A fundamental problem when aggregating Markov chains is the specification of the number of state groups. Too few state groups may fail to sufficiently capture the pertinent dynamics of the original, high-order Markov chain. Too many state…
The conventional perspective on Markov chains considers decision problems concerning the probabilities of temporal properties being satisfied by traces of visited states. However, consider the following query made of a stochastic system…
Many problems of practical interest rely on Continuous-time Markov chains~(CTMCs) defined over combinatorial state spaces, rendering the computation of transition probabilities, and hence probabilistic inference, difficult or impossible…
We consider a model for a queue in which only a fixed number $N$ of customers can join. Each customer joins the queue independently at an exponentially distributed time. Assuming further that the service times are independent and follow an…
Continuous-time Markov chains describing interacting processes exhibit a state space that grows exponentially in the number of processes. This state-space explosion renders the computation or storage of the time-marginal distribution, which…
We consider reversible ergodic Markov chains with finite state space, and we introduce a new notion of quasi-stationary distribution that does not require the presence of any absorbing state. In our setting, the hitting time of the…
In this note we consider a Markov chain formed by a finite system of interacting birth-and-death processes on a finite state space. We study an asymptotic behaviour of the Markov chain as its state space becomes large. In particular, we…
A discrete-time stochastic process derived from a model of basketball is used to generalize any discrete distribution. The generalized distributions can have one or two more parameters than the parent distribution. Those derived from…
In this paper we study a reflected Markov-modulated Brownian motion with a two sided reflection in which the drift, diffusion coefficient and the two boundaries are (jointly) modulated by a finite state space irreducible continuous time…
A new object of the probability theory, two-sided chain of events (symbols), is introduced. A theory of multi-steps Markov chains with long-range memory, proposed earlier in Phys. Rev. E 68, 06117 (2003), is developed and used to establish…
We study the synchronization behavior of discrete-time Markov chains on countable state spaces. Representing a Markov chain in terms of a random dynamical system, which describes the collective dynamics of trajectories driven by the same…
We consider a Markov chain of point processes such that each state is a super position of an independent cluster process with the previous state as its centre process together with some independent noise process. The model extends earlier…
We study random walk on complex networks with transition probabilities which depend on the current and previously visited nodes. By using an absorbing Markov chain we derive an exact expression for the mean first passage time between pairs…
The paper studies the time-homogeneous two-state Markov chains; the states are assumed to be binary symbols 0 and 1. The higher-order absolute differences taken from progressive states of a given chain are considered. A discrete capacity of…
We study positive recurrence and transience of a two-station network in which the behavior of the server in each station is governed by a Markov chain with a finite number of server states; this service process can represent various service…
A Markov process fluctuating away from its typical behavior can be represented in the long-time limit by another Markov process, called the effective or driven process, having the same stationary states as the original process conditioned…
We analyze opportunistic schemes for transmission scheduling from one of $n$ homogeneous queues whose channel states fluctuate independently. Considered schemes consist of the LCQ policy, which transmits from a longest connected queue in…
Markov cohort state-transition models have been the standard approach for simulating the prognosis of patients or, more generally, the life trajectories of individuals over a time period. Current approaches for estimating the variance of a…
This paper addresses the problem of finding a minimum-cost $m$-state Markov chain $(S_0,\ldots,S_{m-1})$ in a large set of chains. The chains studied have a reward associated with each state. The cost of a chain is its "gain", i.e., its…