Related papers: Loop-free Markov chains as determinantal point pro…
We consider an infinite horizon optimal control problem for a continuous-time Markov chain $X$ in a finite set $I$ with noise-free partial observation. The observation process is defined as $Y_t = h(X_t)$, $t \geq 0$, where $h$ is a given…
We give computable bounds on the rate of convergence of the transition probabilities to the stationary distribution for a certain class of geometrically ergodic Markov chains. Our results are different from earlier estimates of Meyn and…
We investigate recurrence and transience of Branching Markov Chains (BMC) in discrete time. Branching Markov Chains are clouds of particles which move (according to an irreducible underlying Markov Chain) and produce offspring…
In this paper we study the central limit theorem for additive functionals of stationary Markov chains with general state space by using a new idea involving conditioning with respect to both the past and future of the chain. Practically, we…
Scaled type Markov renewal processes generalize classical renewal processes: renewal times come from a one parameter family of probability laws and the sequence of the parameters is the trajectory of an ergodic Markov chain. Our primary…
Verification of infinite-state Markov chains is still a challenge despite several fruitful numerical or statistical approaches. For decisive Markov chains, there is a simple numerical algorithm that frames the reachability probability as…
For Markov processes evolving on multiple time-scales a combination of large component scalings and averaging of rapid fluctuations can lead to useful limits for model approximation. A general approach to proving a law of large numbers to a…
We explore two notions of stationary processes. The first is called a random-step Markov process in which the stationary process of states, $(X_i)_{i \in \mathbb{Z}}$ has a stationary coupling with an independent process on the positive…
We analyze the notions of monotonicity and complete monotonicity for Markov Chains in continuous-time, taking values in a finite partially ordered set. Similarly to what happens in discrete-time, the two notions are not equivalent. However,…
We study linear statistics of a class of determinantal processes which interpolate between Poisson and GUE/Ginibre statistics in dimension 1 or 2. These processes are obtained by performing an independent Bernoulli percolation on the…
Sampling from a random discrete distribution induced by a `stick-breaking' process is considered. Under a moment condition, it is shown that the asymptotics of the sequence of occupancy numbers, and of the small-parts counts (singletons,…
A time-dependent finite-state Markov chain that uses doubly stochastic transition matrices, is considered. Entropic quantities that describe the randomness of the probability vectors, and also the randomness of the discrete paths, are…
We propose distribution-free runs-based control charts for detecting location shifts. Using the fact that given the number of total successes, the outcomes of a sequence of Bernoulli trials are random permutations, we are able to control…
We define the concept of an "open" Markov process, or more precisely, continuous-time Markov chain, which is one where probability can flow in or out of certain states called "inputs" and "outputs". One can build up a Markov process from…
In this paper, we prove maximal inequalities and study the functional central limit theorem for the partial sums of linear processes generated by dependent innovations. Due to the general weights, these processes can exhibit long-range…
The main objective of this paper is to establish bootstrap uniform functional central limit theorem for Harris recurrent Markov chains over uniformly bounded classes of functions. We show that the result can be generalized also to the…
We study a catalytic branching process (CBP) with any finite set of catalysts. This model describes a system of particles where the movement is governed by a Markov chain with arbitrary finite or countable state space and the branching may…
We revisit closed-loop performance guarantees for Model Predictive Control in the deterministic and stochastic cases, which extend to novel performance results applicable to receding horizon control of Partially Observable Markov Decision…
Let $A$ be a transition probability kernel on a finite state space $\Delta^o =\{1, \ldots , d\}$ such that $A(x,y)>0$ for all $x,y \in \Delta^o$. Consider a reinforced chain given as a sequence $\{X_n, \; n \in \mathbb{N}_0\}$ of…
We use results from zero-error information theory to determine the set of non-injective functions through which a Markov chain can be projected without losing information. These lumping functions can be found by clique partitioning of a…