Related papers: Stationary analysis of certain Markov-modulated re…
In this paper we study the state-feedback stabilization of a discrete-time Markov jump linear system when the observation of the Markov chain of the system, called the Markov state, is time-randomized by another Markov chain. Embedding the…
Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…
A discrete-time two-dimensional quasi-birth-and-death process (2d-QBD process), $\{{\boldsymbol{Y}}_n\}=\{(X_{1,n},X_{2,n},J_n)\}$, is a two-dimensional skip-free random walk $\{(X_{1,n},X_{2,n})\}$ on $\mathbb{Z}_+^2$ with a supplemental…
Graphical Models have various applications in science and engineering which include physics, bioinformatics, telecommunication and etc. Usage of graphical models needs complex computations in order to evaluation of marginal functions,so…
In this paper we study a class of Quasi-Skipfree (QSF) processes where the transition rate submatrices in the skipfree direction have a column times row structure. Under homogeneity and irreducibility assumptions we show that the stationary…
We consider a Markov jump process on a general state space to which we apply a time-dependent weak perturbation over a finite time interval. By martingale-based stochastic calculus, under a suitable exponential moment bound for the…
Mathematical models of motility are often based on random-walk descriptions of discrete individuals that can move according to certain rules. It is usually the case that large masses concentrated in small regions of space have a great…
A probabilistic Markov Chain (MC) surrogate model for a two-dimensional system of interacting particles within a square domain having inherent symmetries is developed. Particles are assumed to be circular and identical, colliding with each…
We study a linear recursion with random Markov-dependent coefficients. In a "regular variation in, regular variation out" setup we show that its stationary solution has a multivariate regularly varying distribution. This extends results…
A random walk is a basic stochastic process on graphs and a key primitive in the design of distributed algorithms. One of the most important features of random walks is that, under mild conditions, they converge to a stationary distribution…
In this paper, we consider a general class of two-time-scale Markov chains whose transition rate matrix depends on a parameter $\lambda>0$. We assume that some transition rates of the Markov chain will tend to infinity as…
In the study of dynamical processes on networks, there has been intense focus on network structure -- i.e., the arrangement of edges and their associated weights -- but the effects of the temporal patterns of edges remains poorly…
In this paper we study the dynamics of nonlinear random walks. While typical random walks on networks consist of standard Markov chains whose static transition probabilities dictate the flow of random walkers through the network, nonlinear…
A study of time homogeneous, real valued Markov processes with a special property and a non-atomic initial distribution is provided. The new notion of a function of evolution of distribution which determines the dependency between one…
We examine reaction networks (CRNs) through their associated continuous-time Markov processes. Studying the dynamics of such networks is in general hard, both analytically and by simulation. In particular, stationary distributions of…
We analyze opportunistic schemes for transmission scheduling from one of $n$ homogeneous queues whose channel states fluctuate independently. Considered schemes consist of the LCQ policy, which transmits from a longest connected queue in…
In this paper, we consider a subclass of piecewise deterministic Markov processes with a Polish state space that involve a deterministic motion punctuated by random jumps, occurring in a Poisson-like fashion with some state-dependent rate,…
We consider a stable open queuing network as a steady non-equilibrium system of interacting particles. The network is completely specified by its underlying graphical structure, type of interaction at each node, and the Markovian transition…
In this paper we analyze an $M/M/1$ queueing system with an arbitrary number of customer classes, with class-dependent exponential service rates and preemptive priorities between classes. The queuing system can be described by a…
These lecture notes introduce the statistical analysis of continuous-time generative models built from Markov dynamics. We begin with the stochastic-calculus foundations of score-based diffusion models, including time reversal, score…