Related papers: When memory pays: Discord in hidden Markov models

Multi-state models are commonly used for intermittent observations of a state over time, but these are generally based on the Markov assumption, that transition rates are independent of the time spent in current and previous states. In a…

Partially observable Markov decision processes (POMDPs) model specific environments in sequential decision-making under uncertainty. Critically, optimal policies for POMDPs may not be robust against perturbations in the environment.…

When learning a hidden Markov model (HMM), sequen- tial observations can often be complemented by real-valued summary response variables generated from the path of hid- den states. Such settings arise in numerous domains, includ- ing many…

We consider a unified framework of sequential change-point detection and hypothesis testing modeled by means of hidden Markov chains. One observes a sequence of random variables whose distributions are functionals of a hidden Markov chain.…

Reactive (memoryless) policies are sufficient in completely observable Markov decision processes (MDPs), but some kind of memory is usually necessary for optimal control of a partially observable MDP. Policies with finite memory can be…

A statistical test is presented to decide whether data are adequately described by probabilistic functions of finite state Markov chains (''hidden Markov models'') as applied in the analysis of ion channel data. Particularly, the test can…

Markovian-regime-switching (MRS) models are commonly used for modelling economic time series, including electricity prices where independent regime models are used, since they can more accurately and succinctly capture electricity price…

The Markov-modulated Poisson process is utilised for count modelling in a variety of areas such as queueing, reliability, network and insurance claims analysis. In this paper, we extend the Markov-modulated Poisson process framework through…

Machine learning (ML) systems are increasingly deployed in high-stakes domains where reliability is paramount. This thesis investigates how uncertainty estimation can enhance the safety and trustworthiness of ML, focusing on selective…

Time series subject to change in regime have attracted much interest in domains such as econometry, finance or meteorology. For discrete-valued regimes, some models such as the popular Hidden Markov Chain (HMC) describe time series whose…

We present a new algorithm for identifying the transition and emission probabilities of a hidden Markov model (HMM) from the emitted data. Expectation-maximization becomes computationally prohibitive for long observation records, which are…

We consider the estimation of the transition matrix of a hidden Markovian process by using information geometry with respect to transition matrices. In this paper, only the histogram of $k$-memory data is used for the estimation. To…

The optimality of Bayesian filtering relies on the completeness of prior models, while deep learning holds a distinct advantage in learning models from offline data. Nevertheless, the current fusion of these two methodologies remains…

Runtime monitors assess whether a system is in an unsafe state based on a stream of observations. We study the problem where the system is subject to probabilistic uncertainty and described by a hidden Markov model. A stream of observations…

In order to steer e-commerce users towards making a purchase, marketers rely upon predictions of when users exit without purchasing. Previously, such predictions were based upon hidden Markov models (HMMs) due to their ability of modeling…

In order to evaluate and compare different recursive filters, simulation is a common tool and numerous simulation models are widely used as 'benchmark'. In the simulation, the continuous time dynamic system is converted into a discrete-time…

This paper presents new theory and methodology for the Bayesian estimation of overfitted hidden Markov models, with finite state space. The goal is then to achieve posterior emptying of extra states. A prior configuration is constructed…

A penalized maximum likelihood estimation approach is proposed for discrete-time hidden Markov models where covariates affect the observed responses and serial dependence is considered. The proposed penalized maximum likelihood method…

Data collected from wearable devices and smartphones can shed light on an individual's pattern of behavioral and circadian routine. Phone use can be modeled as alternating event process, between the state of active use and the state of…

Recent studies have proposed that one can summarize brain activity into dynamics among a relatively small number of hidden states and that such an approach is a promising tool for revealing brain function. Hidden Markov models (HMMs) are a…