Related papers: Trajectory composition of Poisson time changes and…
A $M/M/1$ queue with catastrophes is a modified $M/M/1$ queue model for which, according to the times of a Poisson process, catastrophes occur leaving the system empty. In this work, we study a fractional $M/M/1$ queue with catastrophes,…
We consider an exhaustive polling system with three nodes in its transient regime under a switching rule of generalized greedy type. We show that, for the system with Poisson arrivals and service times with finite second moment, the…
A Markovian bridge is a probability measure taken from a disintegration of the law of an initial part of the path of a Markov process given its terminal value. As such, Markovian bridges admit a natural parameterization in terms of the…
In this paper, we provide a methodology for computing the probability distribution of sojourn times for a wide class of Markov chains. Our methodology consists in writing out linear systems and matrix equations for generating functions…
We classify the rare events of structured, memoryful stochastic processes and use this to analyze sequential and parallel generators for these events. Given a stochastic process, we introduce a method to construct a new process whose…
The goal of this work is to formally abstract a Markov process evolving in discrete time over a general state space as a finite-state Markov chain, with the objective of precisely approximating its state probability distribution in time,…
In this work, we present a general method to establish properties of multi-dimensional continuous-time Markov chains representing stochastic reaction networks. This method consists of grouping states together (via a partition of the state…
We construct a family of processes, from a single Poisson process, that converges in law to a complex Brownian motion. Moreover, we find realizations of these processes that converge almost surely to the complex Brownian motion, uniformly…
The solution of Poisson's equation plays a key role in constructing the martingale through which sums of Markov correlated random variables can be analyzed. In this paper, we study two different representations for the solution in countable…
Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…
In this paper we develop a method to compute the solution to a countable (finite or infinite) set of equations that occurs in many different fields including Markov processes that model queueing systems, birth-and-death processes and…
In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with…
Interpreting partial information collected from systems subject to noise is a key problem across scientific disciplines. Theoretical frameworks often focus on the dynamics of variables that result from coarse-graining the internal states of…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
We construct a family of self-similar Markov martingales with given marginal distributions. This construction uses the self-similarity and Markov property of a reference process to produce a family of Markov processes that possess the same…
Recently Belopolskaya and Suhov studied Markov process's in a random environment, where the environment changes in likeness to a Markov process. Constructions were made to allow the process to "interact with the environment", this was done…
We study quantitative recurrence to rare events in Countable Markov Shifts with recurrent potentials, focusing on return-time statistics to natural target sets for every point. In the positive recurrent case, return-time processes…
Advances in data collection are producing growing volumes of temporal count observations, making adapted modeling increasingly necessary. In this work, we introduce a generative framework for independent component analysis of temporal count…
A birth-death process is a continuous-time Markov chain that counts the number of particles in a system over time. In the general process with $n$ current particles, a new particle is born with instantaneous rate $\lambda_n$ and a particle…
Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…