Related papers: Regeneration-enriched Markov processes with applic…

Performing numerical integration when the integrand itself cannot be evaluated point-wise is a challenging task that arises in statistical analysis, notably in Bayesian inference for models with intractable likelihood functions. Markov…

In this paper, we show the convergence rates of posterior distributions of the model dynamics in a MDP for both episodic and continuous tasks. The theoretical results hold for general state and action space and the parameter space of the…

The paper studies an improved estimate for the rate of convergence for nonlinear homogeneous discrete-time Markov chains. These processes are nonlinear in terms of the distribution law. Hence, the transition kernels are dependent on the…

The Markov Chain Monte Carlo method is at the heart of efficient approximation schemes for a wide range of problems in combinatorial enumeration and statistical physics. It is therefore very natural and important to determine whether…

Particle Markov Chain Monte Carlo methods are used to carry out inference in non-linear and non-Gaussian state space models, where the posterior density of the states is approximated using particles. Current approaches usually perform…

Learning to sample from complex unnormalized distributions is a fundamental challenge in computational physics and machine learning. While score-based and variational methods have achieved success in continuous domains, extending them to…

In the present paper, we consider random invariant densities and the mean ergodic theorem for Markov operator cocycles which are applicable to quenched type random dynamical systems. We give necessary and sufficient conditions for the…

We introduce a new micro-macro Markov chain Monte Carlo method (mM-MCMC) to sample invariant distributions of molecular dynamics systems that exhibit a time-scale separation between the microscopic (fast) dynamics, and the macroscopic…

Markov chain Monte Carlo (MCMC) algorithms are used to estimate features of interest of a distribution. The Monte Carlo error in estimation has an asymptotic normal distribution whose multivariate nature has so far been ignored in the MCMC…

Statistical inference in evolutionary models with site-dependence is a long-standing challenge in phylogenetics and computational biology. We consider the problem of approximating marginal sequence likelihoods under dependent-site models of…

This paper compiles several aspects of the dynamics of stochastic approximation algorithms with Markov iterate-dependent noise when the iterates are not known to be stable beforehand. We achieve the same by extending the lock-in probability…

Population dynamics are often subject to random independent changes in the environment. For the two strategy stochastic replicator dynamic, we assume that stochastic changes in the environment replace the payoffs and variance. This is…

Markov processes restarted or reset at random times to a fixed state or region in space have been actively studied recently in connection with random searches, foraging, and population dynamics. Here we study the large deviations of…

Real-world autonomous systems operate under uncertainty about both their pose and dynamics. Autonomous control systems must simultaneously perform estimation and control tasks to maintain robustness to changing dynamics or modeling errors.…

Designing efficient learning algorithms with complexity guarantees for Markov decision processes (MDPs) with large or continuous state and action spaces remains a fundamental challenge. We address this challenge for entropy-regularized MDPs…

We formally introduce and study locally-balanced Markov jump processes (LBMJPs) defined on a general state space. These continuous-time stochastic processes with a user-specified limiting distribution are designed for sampling in settings…

This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…

We study a general class of random walks driven by a uniquely ergodic Markovian environment. Under a coupling condition on the environment we obtain strong ergodicity properties for the environment as seen from the position of the walker,…

In order to sample from a given target distribution (often of Gibbs type), the Monte Carlo Markov chain method consists in constructing an ergodic Markov process whose invariant measure is the target distribution. By sampling the Markov…

We prove finite sample complexities for sequential Monte Carlo (SMC) algorithms which require only local mixing times of the associated Markov kernels. Our bounds are particularly useful when the target distribution is multimodal and global…