Related papers: Differential equation approximations for Markov ch…
We consider continuous-space, discrete-time Markov chains on $\mathbb{R}^d$, that admit a finite number $N$ of metastable states. Our main motivation for investigating these processes is to analyse random Poincar\'e maps, which describe…
This paper is a survey of various proofs of the so called {\em fundamental theorem of Markov chains}: every ergodic Markov chain has a unique positive stationary distribution and the chain attains this distribution in the limit independent…
This paper considers a new approach to using Markov chain Monte Carlo (MCMC) in contexts where one may adopt multilevel (ML) Monte Carlo. The underlying problem is to approximate expectations w.r.t. an underlying probability measure that is…
This paper gives a method for computing distributions associated with patterns in the state sequence of a hidden Markov model, conditional on observing all or part of the observation sequence. Probabilities are computed for very general…
We develop a framework for the compression of reversible Markov chains with rigorous error control. Given a subset of selected states, we construct reduced dynamics that can be lifted to an approximation of the full dynamics, and we prove…
Markov chains and Markov decision processes (MDPs) are well-established probabilistic models. While finite Markov models are well-understood, analysing their infinite counterparts remains a significant challenge. Decisiveness has proven to…
In this invited contribution, we revisit the stochastic shortest path problem, and show how recent results allow one to improve over the classical solutions: we present algorithms to synthesize strategies with multiple guarantees on the…
We consider the problem of selecting deterministic or stochastic models for a biological, ecological, or environmental dynamical process. In most cases, one prefers either deterministic or stochastic models as candidate models based on…
In this paper, an approach to estimating a nonlinear deterministic model is presented. We introduce a stochastic model with extremely small variances so that the deterministic and stochastic models are essentially indistinguishable from…
This paper generalizes the notion of stochastic order to a relation between probability measures over arbitrary measurable spaces. This generalization is motivated by the observation that for the stochastic ordering of two stationary Markov…
In this paper a new version of the chain rule for calculating the mean square derivative of a second-order stochastic process is proven. This random operational calculus rule is applied to construct a rigorous mean square solution of the…
Consider a finite irreducible Markov chain with invariant distribution $\pi$. We use the inner product induced by $\pi$ and the associated heat operator to simplify and generalize some results related to graph partitioning and the small-set…
The rigorous linking of exact stochastic models to mean-field approximations is studied. Starting from the differential equation point of view the stochastic model is identified by its Kolmogorov equations, which is a system of linear ODEs…
A discrete-time stochastic process derived from a model of basketball is used to generalize any discrete distribution. The generalized distributions can have one or two more parameters than the parent distribution. Those derived from…
A common tool in the practice of Markov Chain Monte Carlo is to use approximating transition kernels to speed up computation when the desired kernel is slow to evaluate or intractable. A limited set of quantitative tools exist to assess the…
We consider a class of non-homogeneous Markov chains, that contains many natural examples. Next, using martingale methods, we establish some deviation and moment inequalities for separately Lipschitz functions of such a chain, under moment…
Stochastic kernel based dimensionality reduction approaches have become popular in the last decade. The central component of many of these methods is a symmetric kernel that quantifies the vicinity between pairs of data points and a…
The paper considers a stochastic differential equation of Duffing type with Markov coefficients. The existence of unpredictable solutions is considered. The unpredictability is a property of bounded functions characterized by unbounded…
We propose a method to sample stationary properties of solutions of stochastic differential equations, which is accurate and efficient if there are rarely visited regions or rare transitions between distinct regions of the state space. The…
In this paper, we develop necessary and sufficient conditions for the validity of a martingale approximation for the partial sums of a stationary process in terms of the maximum of consecutive errors. Such an approximation is useful for…