Related papers: Markov Chain Variance Estimation: A Stochastic App…

We introduce computational methods that allow for effective estimation of a flexible, parametric non-stationary spatial model when the field size is too large to compute the multivariate normal likelihood directly. In this method, the field…

We consider the problem of parameter estimation for a stochastic McKean-Vlasov equation, and the associated system of weakly interacting particles. We study two cases: one in which we observe multiple independent trajectories of the…

In this paper, we present a methodology to estimate the parameters of stochastically contaminated models under two contamination regimes. In both regimes, we assume that the original process is a variable length Markov chain that is…

We consider the efficient use of an approximation within Markov chain Monte Carlo (MCMC), with subsequent importance sampling (IS) correction of the Markov chain inexact output, leading to asymptotically exact inference. We detail…

Markov parameters play a key role in system identification. There exists many algorithms where these parameters are estimated using least-squares in a first, pre-processing, step, including subspace identification and multi-step…

We present a simulation methodology for Bayesian estimation of rate parameters in Markov jump processes arising for example in stochastic kinetic models. To handle the problem of missing components and measurement errors in observed data,…

We consider stochastic optimization problems where data is drawn from a Markov chain. Existing methods for this setting crucially rely on knowing the mixing time of the chain, which in real-world applications is usually unknown. We propose…

In this paper, we consider the problem of online asymptotic variance estimation for particle filtering and smoothing. Current solutions for the particle filter rely on the particle genealogy and are either unstable or hard to tune in…

We consider a unified framework of sequential change-point detection and hypothesis testing modeled by means of hidden Markov chains. One observes a sequence of random variables whose distributions are functionals of a hidden Markov chain.…

A classical approach for approximating expectations of functions w.r.t. partially known distributions is to compute the average of function values along a trajectory of a Metropolis-Hastings (MH) Markov chain. A key part in the MH algorithm…

In this paper, we provide a new algorithm for the problem of prediction in Reinforcement Learning, \emph{i.e.}, estimating the Value Function of a Markov Reward Process (MRP) using the linear function approximation architecture, with memory…

We consider the stochastic approximation problem where a convex function has to be minimized, given only the knowledge of unbiased estimates of its gradients at certain points, a framework which includes machine learning methods based on…

This paper studies the exponential stability of random matrix products driven by a general (possibly unbounded) state space Markov chain. It is a cornerstone in the analysis of stochastic algorithms in machine learning (e.g. for parameter…

Motivated by techniques developed in recent progress on lower bounds for sublinear time algorithms (Behnezhad, Roghani and Rubinstein, STOC 2023, FOCS 2023, and STOC 2024) we introduce and study a new class of randomized algorithmic…

In this paper, we investigate a nonparametric approach to provide a recursive estimator of the transition density of a non-stationary piecewise-deterministic Markov process, from only one observation of the path within a long time. In this…

Maximum likelihood estimation of large Markov-switching vector autoregressions (MS-VARs) can be challenging or infeasible due to parameter proliferation. To accommodate situations where dimensionality may be of comparable order to or…

We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes…

A Markov-switching observation-driven model is a stochastic process $((S_t,Y_t))_{t \in \mathbb{Z}}$ where $(S_t)_{t \in \mathbb{Z}}$ is an unobserved Markov chain on a finite set and $(Y_t)_{t \in \mathbb{Z}}$ is an observed stochastic…

We establish non-asymptotic error bounds for the classical Maximal Likelihood Estimation of the transition matrix of a given Markov chain. Meanwhile, in the reversible case, we propose a new reversibility-preserving online Symmetric…

This article surveys recent advancements of strategy designs for persistent robotic surveillance tasks with the focus on stochastic approaches. The problem describes how mobile robots stochastically patrol a graph in an efficient way where…