Related papers: Filters and smoothers for self-exciting Markov mod…

We consider the modeling of data generated by a latent continuous-time Markov jump process with a state space of finite but unknown dimensions. Typically in such models, the number of states has to be pre-specified, and Bayesian inference…

This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…

We investigate the increase in efficiency of simulated and parallel tempering MCMC algorithms when using non-reversible updates to give them "momentum". By making a connection to a certain simple discrete Markov chain, we show that, under…

We develop a new methodology for the fluctuation theory of continuous-time skip-free Markov chains, extending the recent work of Choi and Patie [5] for discrete-time skip-free Markov chains. As the main application we use it to derive a…

We solve a linear quadratic optimal control problem for sampled-data systems with stochastic delays. The delays are stochastically determined by the last few delays. The proposed optimal controller can be efficiently computed by iteratively…

Many problems in quantum information theory can be formulated as optimizations over the sequential outcomes of dynamical systems subject to unpredictable external influences. Such problems include many-body entanglement detection through…

This paper presents a novel algorithm for efficient online estimation of the filter derivatives in general hidden Markov models. The algorithm, which has a linear computational complexity and very limited memory requirements, is furnished…

We construct a non-decreasing pure jump Markov process, whose jump measure heavily depends on the values taken by the process. We determine the singularity spectrum of this process, which turns out to be random and to depend locally on the…

Markov switching models are a popular family of models that introduces time-variation in the parameters in the form of their state- or regime-specific values. Importantly, this time-variation is governed by a discrete-valued latent…

In a previous paper by the second author,two Markov chain Monte Carlo perfect sampling algorithms -- one called coupling from the past (CFTP) and the other (FMMR) based on rejection sampling -- are compared using as a case study the…

We consider the control of a Markov decision process (MDP) that undergoes an abrupt change in its transition kernel (mode). We formulate the problem of minimizing regret under control-switching based on mode change detection, compared to a…

We study the forgetting properties of the particle filter when its state - the collection of particles - is regarded as a Markov chain. Under a strong mixing assumption on the particle filter's underlying Feynman-Kac model, we find that the…

This paper investigates methods for estimating the optimal stochastic control policy for a Markov Decision Process with unknown transition dynamics and an unknown reward function. This form of model-free reinforcement learning comprises…

This paper deals with the design of discrete-time algorithms for the robust filtering differentiator. Two discrete-time realizations of the filtering differentiator are introduced. The first one, which is based on an exact discretization of…

Many events occur in the world. Some event types are stochastically excited or inhibited---in the sense of having their probabilities elevated or decreased---by patterns in the sequence of previous events. Discovering such patterns can help…

Lumping a Markov process introduces a coarser level of description that is useful in many contexts and applications. The dynamics on the coarse grained states is often approximated by its Markovian component. In this letter we derive…

We introduce discrete time Markov chains that preserve uniform measures on boxed plane partitions. Elementary Markov steps change the size of the box from (a x b x c) to ((a-1) x (b+1) x c) or ((a+1) x (b-1) x c). Algorithmic realization of…

Events in the world may be caused by other, unobserved events. We consider sequences of events in continuous time. Given a probability model of complete sequences, we propose particle smoothing---a form of sequential importance…

We consider the estimation of parameters encoded in the measurement record of a continuously monitored quantum system in the jump unraveling, corresponding to a single-shot scenario, where information is continuously gathered. Here, it is…

In this paper we consider filtering and smoothing of partially observed chaotic dynamical systems that are discretely observed, with an additive Gaussian noise in the observation. These models are found in a wide variety of real…