Related papers: Sampling and Filtering with Markov Chains

Recently there have been exciting developments in Monte Carlo methods, with the development of new MCMC and sequential Monte Carlo (SMC) algorithms which are based on continuous-time, rather than discrete-time, Markov processes. This has…

In this work, we extend the idea of Quantum Markov chains [S. Gudder. Quantum Markov chains. J. Math. Phys., 49(7), 2008] in order to propose Quantum Hidden Markov Models (QHMMs). For that, we use the notions of Transition Operation…

The problem of estimating an unknown discrete distribution from its samples is a fundamental tenet of statistical learning. Over the past decade, it attracted significant research effort and has been solved for a variety of divergence…

Markov chain Monte Carlo (MCMC) is a sampling-based method for estimating features of probability distributions. MCMC methods produce a serially correlated, yet representative, sample from the desired distribution. As such it can be…

Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models…

Parameter estimation for discretely observed Markov processes is a challenging problem. However, simulation of Markov processes is straightforward using the Gillespie algorithm. We exploit this ease of simulation to develop an effective…

In recent work (arXiv:1006.3100v1), we have presented a novel approach for improving particle filters for multi-target tracking. The suggested approach was based on drift homotopy for stochastic differential equations. Drift homotopy was…

We introduce the Hamming Ball Sampler, a novel Markov Chain Monte Carlo algorithm, for efficient inference in statistical models involving high-dimensional discrete state spaces. The sampling scheme uses an auxiliary variable construction…

Markov jump processes (or continuous-time Markov chains) are a simple and important class of continuous-time dynamical systems. In this paper, we tackle the problem of simulating from the posterior distribution over paths in these models,…

Advances in digital sensors, digital data storage and communications have resulted in systems being capable of accumulating large collections of data. In the light of dealing with the challenges that massive data present, this work proposes…

Atomistic simulations provide valuable insights into the physical processes governing material behavior. However, their applicability is fundamentally constrained by the limited time scales accessible to brute-force simulations. This…

Hidden Markov Models (HMMs) are powerful tools for modeling sequential data, where the underlying states evolve in a stochastic manner and are only indirectly observable. Traditional HMM approaches are well-established for linear sequences,…

Main objects of the paper are stationary and weak KAM Hamilton-Jacobi equations on the finite-dimensional torus. The key idea of the paper is to replace the underlying calculus of variations problems with continuous time Markov decision…

This paper considers the optimal control of time varying continuous time Markov chains whose transition rates are themselves Markov processes. In one set of problems the solution of an ordinary differential equation is shown to determine…

The analysis of many problems of interest associated with Markov chains, e.g. stationary distributions, moments of first passage time distributions and moments of occupation time random variables, involves the solution of a system of linear…

We consider a unified framework of sequential change-point detection and hypothesis testing modeled by means of hidden Markov chains. One observes a sequence of random variables whose distributions are functionals of a hidden Markov chain.…

Probabilistic model checking for systems with large or unbounded state space is a challenging computational problem in formal modelling and its applications. Numerical algorithms require an explicit representation of the state space, while…

An introduction to the use of linchpin variables in Markov chain Monte Carlo (MCMC) is provided. Before the widespread adoption of MCMC methods, conditional sampling using linchpin variables was essentially the only practical approach for…

Bayesian filtering aims at tracking sequentially a hidden process from an observed one. In particular, sequential Monte Carlo (SMC) techniques propagate in time weighted trajectories which represent the posterior probability density…

Conformal prediction is a widely used method to quantify the uncertainty of a classifier under the assumption of exchangeability (e.g., IID data). We generalize conformal prediction to the Hidden Markov Model (HMM) framework where the…