Related papers: Maximum Likelihood Estimation for Markov Chains

We present two data-driven procedures to estimate the transition density of an homogeneous Markov chain. The first yields to a piecewise constant estimator on a suitable random partition. By using an Hellinger-type loss, we establish…

A rescaled Markov chain converges uniformly in probability to the solution of an ordinary differential equation, under carefully specified assumptions. The presentation is much simpler than those in the outside literature. The result may be…

We propose a new Markov chain Monte Carlo method in which trial configurations are generated by evolving a state, sampled from a prior distribution, using a Markov transition matrix. We present two prototypical algorithms and derive their…

We review recent results on the metastable behavior of continuous-time Markov chains derived through the characterization of Markov chains as unique solutions of martingale problems.

Improved rates of convergence for ergodic homogeneous Markov chains are studied. In comparison to the earlier papers the setting is also generalised to the case without a unique dominated measure. Examples are provided where the new bound…

We introduce bounds on the finite-time performance of Markov chain Monte Carlo algorithms in approaching the global solution of stochastic optimization problems over continuous domains. A comparison with other state-of-the-art methods…

Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…

In this paper we consider estimation of sparse covariance matrices and propose a thresholding procedure which is adaptive to the variability of individual entries. The estimators are fully data driven and enjoy excellent performance both…

This paper is dedicated to the investigation of a new numerical method to approximate the optimal stopping problem for a discrete-time continuous state space Markov chain under partial observations. It is based on a two-step discretization…

We study a recently introduced gradient-based Markov chain Monte Carlo method based on 'Barker dynamics'. We provide a full derivation of the method from first principles, placing it within a wider class of continuous-time Markov jump…

Many modern techniques employed in physics, such a computation of path integrals, rely on random walks on graphs that can be represented as Markov chains. Traditionally, estimates of running times of such sampling algorithms are computed…

We analyse the structure of imprecise Markov chains and study their convergence by means of accessibility relations. We first identify the sets of states, so-called minimal permanent classes, that are the minimal sets capable of containing…

Stochastic processes are often used to model complex scientific problems in fields ranging from biology and finance to engineering and physical science. This paper investigates rate-optimal estimation of the volatility matrix of a…

Suppose we observe a geometrically ergodic semi-Markov process and have a parametric model for the transition distribution of the embedded Markov chain, for the conditional distribution of the inter-arrival times, or for both. The first two…

We address a problem of covariance selection, where we seek a trade-off between a high likelihood against the number of non-zero elements in the inverse covariance matrix. We solve a maximum likelihood problem with a penalty term given by…

We offer a method to estimate a covariance matrix in the special case that \textit{both} the covariance matrix and the precision matrix are sparse --- a constraint we call double sparsity. The estimation method is maximum likelihood,…

We consider the problem of estimating the asymptotic variance of a function defined on a Markov chain, an important step for statistical inference of the stationary mean. We design a novel recursive estimator that requires $O(1)$…

Based on information theory, we present a method to determine an optimal Markov approximation for modelling and prediction from time series data. The method finds a balance between minimal modelling errors by taking as much as possible…

Markov Chain Monte Carlo (MCMC) algorithms are often used for approximate inference inside learning, but their slow mixing can be difficult to diagnose and the approximations can seriously degrade learning. To alleviate these issues, we…

In this paper, we present a novel iterative Monte Carlo method for approximating the stationary probability of a single state of a positive recurrent Markov chain. We utilize the characterization that the stationary probability of a state…