Related papers: Forgetting of the initial condition for the filter…
This article discusses a partially adapted particle filter for estimating the likelihood of a nonlinear structural econometric state space models whose state transition density cannot be expressed in closed form. The filter generates the…
We seek to learn an effective policy for a Markov Decision Process (MDP) with continuous states via Q-Learning. Given a set of basis functions over state action pairs we search for a corresponding set of linear weights that minimizes the…
Attractiveness is a fundamental tool to study interacting particle systems and the basic coupling construction is a usual route to prove this property, as for instance in the simple exclusion process. We consider here general exclusion…
We introduce a framework for inference in general state-space hidden Markov models (HMMs) under likelihood misspecification. In particular, we leverage the loss-theoretic perspective of Generalized Bayesian Inference (GBI) to define…
Particle Marginal Metropolis-Hastings (PMMH) is a general approach to Bayesian inference when the likelihood is intractable, but can be estimated unbiasedly. Our article develops an efficient PMMH method that scales up better to higher…
Many problems of practical interest rely on Continuous-time Markov chains~(CTMCs) defined over combinatorial state spaces, rendering the computation of transition probabilities, and hence probabilistic inference, difficult or impossible…
An autoregressive process with Markov regime is an autoregressive process for which the regression function at each time point is given by a nonobservable Markov chain. In this paper we consider the asymptotic properties of the maximum…
Quantum Markov models are employed ubiquitously in quantum physics and in quantum information theory due to their relative simplicity and analytical tractability. In particular, these models are known to give accurate approximations for a…
The problem of discrete universal filtering, in which the components of a discrete signal emitted by an unknown source and corrupted by a known DMC are to be causally estimated, is considered. A family of filters are derived, and are shown…
A method is presented for Bayesian model selection without explicitly computing evidences, by using a combined likelihood and introducing an integer model selection parameter $n$ so that Bayes factors, or more generally posterior odds…
Suitable reachability conditions can make two different fixed point semantics of a transition system coincide. For instance, the total and partial expected reward semantics on Markov chains (MCs) coincide whenever the MC at hand is almost…
Order-preserving couplings are elegant tools for obtaining robust estimates of the time-dependent and stationary distributions of Markov processes that are too complex to be analyzed exactly. The starting point of this paper is to study…
State Space Models (SSMs) and Hidden Markov Models (HMMs) are foundational frameworks for modeling sequential data with latent variables and are widely used in signal processing, control theory, and machine learning. Despite their shared…
Conditional particle filters (CPFs) with backward/ancestor sampling are powerful methods for sampling from the posterior distribution of the latent states of a dynamic model such as a hidden Markov model. However, the performance of these…
This paper investigates the stability of Kalman filtering over Gilbert-Elliott channels where random packet drop follows a time-homogeneous two-state Markov chain whose state transition is determined by a pair of failure and recovery rates.…
This paper develops a connection between the asymptotic stability of nonlinear filters and a notion of observability. We consider a general class of hidden Markov models in continuous time with compact signal state space, and call such a…
This article shows how coupled Markov chains that meet exactly after a random number of iterations can be used to generate unbiased estimators of the solutions of the Poisson equation. Through this connection, we re-derive known unbiased…
Developing feature selection algorithms that move beyond a pure correlational to a more causal analysis of observational data is an important problem in the sciences. Several algorithms attempt to do so by discovering the Markov blanket of…
Nonlinear model predictive control has become a popular approach to deal with highly nonlinear and unsteady state systems, the performance of which can however deteriorate due to unaccounted uncertainties. Model predictive control is…
It is hard to identify nonlinear biological models strictly from data, with results that are often sensitive to experimental conditions. Automated experimental workflows and liquid handling enables unprecedented throughput, as well as the…