Related papers: Regenerative processes
It has been established under very general conditions that the ergodic properties of Markov processes are inherited by their conditional distributions given partial information. While the existing theory provides a rather complete picture…
We consider regenerative processes with values in some Polish space. We define their \epsilon-big excursions as excursions e such that f(e)>\epsilon, where f is some given functional on the space of excursions which can be thought of as,…
This paper focuses on optimizing probabilities of events of interest defined over general controlled discrete-time Markov processes. It is shown that the optimization over a wide class of $\omega$-regular properties can be reduced to the…
A notion of conditionally identically distributed (c.i.d.) sequences has been studied as a form of stochastic dependence that is weaker than exchangeability, but is equivalent to exchangeability for stationary sequences. In this article we…
A semi-Markov process is one that changes states in accordance with a Markov chain but takes a random amount of time between changes. We consider the generalisation to semi-Markov processes of the classical Lamperti law for the occupation…
Markov decision processes (MDPs) are a popular model for decision-making in the presence of uncertainty. The conventional view of MDPs in verification treats them as state transformers with probabilities defined over sequences of states and…
We present the observation that the process of stochastic model predictive control can be formulated in the framework of iterated function systems. The latter has a rich ergodic theory that can be applied to study the system's long-run…
This paper gives an elementary proof for the following theorem: a renewal process can be represented by a doubly-stochastic Poisson process (DSPP) if and only if the Laplace-Stieltjes transform of the inter-arrival times is of the following…
Consider a uniformly sampled random $d$-regular graph on $n$ vertices. If $d$ is fixed and $n$ goes to $\infty$ then we can relate typical (large probability) properties of such random graph to a family of invariant random processes (called…
Random walks in random scenery are processes defined by $Z_n:=\sum_{k=1}^n\xi_{X_1+...+X_k}$, where $(X_k,k\ge 1)$ and $(\xi_y,y\in\mathbb Z)$ are two independent sequences of i.i.d. random variables. We suppose that the distributions of…
We give strong bounds for the rate of convergence of the regenerative process distribution to the stationary distribution in the total variation metric. These bounds are obtained by using coupling method. We propose this method for…
Branching processes are models used to describe populations that reproduce and die over time. In the classical setting, an individual's reproductive capacity remains constant throughout its lifetime. However, in real-world situations,…
We employ the framework of multitype Galton-Watson processes to model a population of dividing cells. The cellular type is represented by its biological age, defined as the count of harmful proteins hosted by the cell. The stochastic…
Two independent Poisson streams of jobs flow into a single-server service system having a limited common buffer that can hold at most one job. If a type-i job (i=1,2) finds the server busy, it is blocked and routed to a separate type-i…
Motivated by queueing systems with heterogeneous parallel servers, we consider a class of structured multi-dimensional Markov processes whose state space can be partitioned into two parts: a finite set of boundary states and a structured…
The aim of this short note is to present the notion of IDT processes, which is a wide generalization of L\'{e}vy processes obtained from a modified infinitely divisible property. Special attention is put on a number of examples, in order to…
Statistical Mechanics deals with ensembles of microstates that are compatible with fixed constraints and that on average define a thermodynamic macrostate. The evolution of a small system is normally subjected to changing constraints and…
Normalizing flows, diffusion normalizing flows and variational autoencoders are powerful generative models. This chapter provides a unified framework to handle these approaches via Markov chains. We consider stochastic normalizing flows as…
This paper considers a particular renewal-reward process with multivariate discounted rewards (inputs) where the arrival epochs are adjusted by adding some random delays. Then this accumulated reward can be regarded as multivariate…
The first aim of the present note is to quantify the speed of convergence of a conditioned process toward its Q-process under suitable assumptions on the quasi-stationary distribution of the process. Conversely, we prove that, if a…