Related papers: Markov Infinitely-Divisible Stationary Time-Revers…

We introduce a generalization of the Adaptive Multilevel Splitting algorithm in the discrete time dynamic setting, namely when it is applied to sample rare events associated with paths of Markov chains. By interpreting the algorithm as a…

The goal of this paper is to analyze distributional Markov Decision Processes as a class of control problems in which the objective is to learn policies that steer the distribution of a cumulative reward toward a prescribed target law,…

We study a class of infinite-horizon average-cost Markov Decision Processes (MDPs) whose reward and transition structures are nearly separable. For the totally separable baseline (that is, with no perturbation), we derive an explicit…

The practically important classes of equal-input and of monotone Markov matrices are revisited, with special focus on embeddability, infinite divisibility, and mutual relations. Several uniqueness results for the classic Markov embedding…

Consider a stationary renewal point process on the real line and divide each of the segments it defines in a proportion given by \iid realisations of a fixed distribution $G$ supported by [0,1]. We ask ourselves for which interpoint…

We consider a hidden Markov model with multiple observation processes, one of which is chosen at each point in time by a policy---a deterministic function of the information state---and attempt to determine which policy minimises the…

The infinite (in both directions) sequence of the distributions $\mu^{(k)}$ of the stochastic integrals $\int_0^{\infty-}c^{-N_{t-}^{(k)}} dL_t^{(k)}$ for integers $k$ is investigated. Here $c>1$ and $(N_t^{(k)},L_t^{(k)})$, $t\geq0$, is a…

We introduce a unified operator-theoretic framework for analyzing mixing times of finite-state ergodic Markov chains that applies to both reversible and non-reversible dynamics. The central object in our analysis is the projected transition…

We consider a certain sequence of random walks. The state space of the n-th random walk is the set of all strict partitions of n (that is, partitions without equal parts). We prove that, as n goes to infinity, these random walks converge to…

In this paper, we consider a subclass of piecewise deterministic Markov processes with a Polish state space that involve a deterministic motion punctuated by random jumps, occurring in a Poisson-like fashion with some state-dependent rate,…

Stationary ergodic processes with finite alphabets are estimated by finite memory processes from a sample, an n-length realization of the process, where the memory depth of the estimator process is also estimated from the sample using…

We consider continuous-time Markov chain on a finite state space X. We assume X can be clustered into several subsets such that the intra-transition rates within these subsets are of order $\mathcal{O}(\frac{1}{\epsilon})$ comparing to the…

We analyse the structure of imprecise Markov chains and study their convergence by means of accessibility relations. We first identify the sets of states, so-called minimal permanent classes, that are the minimal sets capable of containing…

We introduce a family of Markov processes on set partitions with a bounded number of blocks, called Lipschitz partition processes. We construct these processes explicitly by a Poisson point process on the space of Lipschitz continuous maps…

The concept of a L\'evy subordinator is generalized to a family of non-decreasing stochastic processes, which are parameterized in terms of two Bernstein functions. Whereas the independent increments property is only maintained in the…

We introduce and study a family of Markov processes on partitions. The processes preserve the so-called z-measures on partitions previously studied in connection with harmonic analysis on the infinite symmetric group. We show that the…

Undirected graphical models, or Markov networks, are a popular class of statistical models, used in a wide variety of applications. Popular instances of this class include Gaussian graphical models and Ising models. In many settings,…

Markov categories have recently turned out to be a powerful high-level framework for probability and statistics. They accommodate purely categorical definitions of notions like conditional probability and almost sure equality, as well as…

We are concerned with the absolute continuity of stationary distributions corresponding to some piecewise deterministic Markov process, being typically encountered in biological models. The process under investigation involves a…

We prove exponential decay of pair correlations for 1D stationary point processes when spacings satisfy a Markov condition, geometric ergodicity, and a condition on exponential moments. The conditions are phrased for stationary sequences of…