Related papers: An approximation theory for Markov chain compressi…

This paper proposes a new type of recurrence where we divide the Markov chains into intervals that start when the chain enters into a subset A, then sample another subset B far away from A and end when the chain again return to A. The…

The spectral gap of a Markov chain can be bounded by the spectral gaps of constituent "restriction" chains and a "projection" chain, and the strength of such a bound is the content of various decomposition theorems. In this paper, we…

A shortcoming of existing reachability approaches for nonlinear systems is the poor scalability with the number of continuous state variables. To mitigate this problem we present a simulation-based approach where we first sample a number of…

We study the computation of lower and upper probabilities of hitting a target set of states for imprecise Markov chains, where transition uncertainty is modelled by a convex set of transition matrices. In the precise case, hitting…

The aim of this paper is to investigate the rebinding effect, a phenomenon describing a "short-time memory" which can occur when projecting a Markov process onto a smaller state space. For guaranteeing a correct mapping by the Markov State…

Compressive sensing is a technique to sample signals well below the Nyquist rate using linear measurement operators. In this paper we present an algorithm for signal reconstruction given such a set of measurements. This algorithm…

We describe several algorithms for matrix completion and matrix approximation when only some of its entries are known. The approximation constraint can be any whose approximated solution is known for the full matrix. For low rank…

Markov chains are an important tool for modelling and evaluating systems in computer science, economics, biology and numerous other fields. Thus, approximating Markov chains is a useful tool for decreasing the computational effort needed…

We consider the problem of testing the identity of a reversible Markov chain against a reference from a single trajectory of observations. Employing the recently introduced notion of a lumping-congruent Markov embedding, we show that, at…

Recently two approximate Newton methods were proposed for the optimisation of Markov Decision Processes. While these methods were shown to have desirable properties, such as a guarantee that the preconditioner is negative-semidefinite when…

Leveraging recent work on data-driven methods for constructing a finite state space Markov process from dynamical systems, we address two problems for obtaining further reduced statistical representations. The first problem is to extract…

We introduce $(\varepsilon, \delta)$-bisimulation, a novel type of approximate probabilistic bisimulation for continuous-time Markov chains. In contrast to related notions, $(\varepsilon, \delta)$-bisimulation allows the use of different…

This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a…

In this paper we develop a statistical estimation technique to recover the transition kernel $P$ of a Markov chain $X=(X_m)_{m \in \mathbb N}$ in presence of censored data. We consider the situation where only a sub-sequence of $X$ is…

Compressed sensing is a theory which guarantees the exact recovery of sparse signals from a small number of linear projections. The sampling schemes suggested by current compressed sensing theories are often of little practical relevance…

The simplest, and most common, stochastic model for population processes, including those from biochemistry and cell biology, are continuous time Markov chains. Simulation of such models is often relatively straightforward as there are…

We consider continuous-time, finite-horizon, optimal quadratic control of semi-Markov jump linear systems (S-MJLS), and develop principled approximations through Markov-like representations for the holding-time distributions. We adopt a…

We prove a functional limit theorem for Markov chains that, in each step, move up or down by a possibly state dependent constant with probability $1/2$, respectively. The theorem entails that the law of every one-dimensional regular…

We prove explicit, i.e., non-asymptotic, error bounds for Markov Chain Monte Carlo methods, such as the Metropolis algorithm. The problem is to compute the expectation (or integral) of f with respect to a measure which can be given by a…

Compressive Sensing (CS) stipulates that a sparse signal can be recovered from a small number of linear measurements, and that this recovery can be performed efficiently in polynomial time. The framework of model-based compressive sensing…