Related papers: Regenerativity of Viterbi process for pairwise Mar…

Interval Markov decision processes (IMDPs) generalise classical MDPs by having interval-valued transition probabilities. They provide a powerful modelling tool for probabilistic systems with an additional variation or uncertainty that…

We consider a pair of correlated processes {Z_n} and {S_n} (two sided), where the former is observable and the later is hidden. The uncertainty in the estimation of Z_n upon its finite past history is H(Z_n|Z_0^{n-1}), and for estimation of…

The Hierarchical Dirichlet Process Hidden Markov Model (HDP-HMM) is a natural Bayesian nonparametric extension of the classical Hidden Markov Model for learning from (spatio-)temporal data. A sticky HDP-HMM has been proposed to strengthen…

This paper studies maintenance optimization for a two-component system under mixed observability. Component~$U_1$ is fully monitored, whereas component~$U_2$ is only partially observable due to sensing limitations. The system exhibits…

We introduce multiple hidden Markov models (MHMMs) where an observed multivariate categorical time series depends on an unobservable multivariate Mar- kov chain. MHMMs provide an elegant framework for specifying various independence…

We introduce a new algorithm for approximate inference that combines reparametrization, Markov chain Monte Carlo and variational methods. We construct a very flexible implicit variational distribution synthesized by an arbitrary Markov…

Experiments, in particular on biological systems, typically probe lower-dimensional observables which are projections of high-dimensional dynamics. In order to infer consistent models capturing the relevant dynamics of the system, it is…

The problem of detection and possible estimation of a signal generated by a dynamic system when a variable number of noisy measurements can be taken is here considered. Assuming a Markov evolution of the system (in particular, the pair…

In a real life process evolving over time, the relationship between its relevant variables may change. Therefore, it is advantageous to have different inference models for each state of the process. Asymmetric hidden Markov models fulfil…

This paper deals with the problem of preventive maintenance (PM) scheduling of pipelines subject to external corrosion defects. The preventive maintenance strategy involves an inspection step at some epoch, together with a repair schedule.…

A labelled Markov decision process (MDP) is a labelled Markov chain with nondeterminism; i.e., together with a strategy a labelled MDP induces a labelled Markov chain. The model is related to interval Markov chains. Motivated by…

This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a…

We present a new method for nonlinear prediction of discrete random sequences under minimal structural assumptions. We give a mathematical construction for optimal predictors of such processes, in the form of hidden Markov models. We then…

We present a data-driven model predictive control scheme for chance-constrained Markovian switching systems with unknown switching probabilities. Using samples of the underlying Markov chain, ambiguity sets of transition probabilities are…

The objective of this article is to study the asymptotic behavior of a new particle filtering approach in the context of hidden Markov models (HMMs). In particular, we develop an algorithm where the latent-state sequence is segmented into…

We present a Metropolis-Hastings Markov chain Monte Carlo (MCMC) algorithm for detecting hidden variables in a continuous time Bayesian network (CTBN), which uses reversible jumps in the sense defined by (Green 1995). In common with several…

In this paper, we are interested in optimal decisions in a partially observable Markov universe. Our viewpoint departs from the dynamic programming viewpoint: we are directly approximating an optimal strategic tree depending on the…

This paper presents new theory and methodology for the Bayesian estimation of overfitted hidden Markov models, with finite state space. The goal is then to achieve posterior emptying of extra states. A prior configuration is constructed…

This paper addresses the trajectory-tracking problem for discrete-time linear time-invariant systems with bounded parametric uncertainty, subject to hard constraints on system states, control inputs, and input rates. Unlike existing…

Parallel tempering (PT) is a class of Markov chain Monte Carlo algorithms that constructs a path of distributions annealing between a tractable reference and an intractable target, and then interchanges states along the path to improve…