Related papers: On the Viterbi process with continuous state space
Consider the problem of predicting the next symbol given a sample path of length n, whose joint distribution belongs to a distribution class that may have long-term memory. The goal is to compete with the conditional predictor that knows…
Isolating slower dynamics from fast fluctuations has proven remarkably powerful, but how do we proceed from partial observations of dynamical systems for which we lack underlying equations? Here, we construct maximally-predictive states by…
This paper considers maximum likelihood (ML) estimation in a large class of models with hidden Markov regimes. We investigate consistency of the ML estimator and local asymptotic normality for the models under general conditions which allow…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a…
This paper investigates MDPs with intermittent state information. We consider a scenario where the controller perceives the state information of the process via an unreliable communication channel. The transmissions of state information…
We introduce a quantum Viterbi decoding algorithm for hidden quantum Markov models (HQMMs) motivated by quantum information processing and quantum algorithms. Given a finite sequence of measurement outcomes, the algorithm identifies hidden…
Many processes, such as discrete event systems in engineering or population dynamics in biology, evolve in discrete space and continuous time. We consider the problem of optimal decision making in such discrete state and action space…
For a Markov decision process with countably infinite states, the optimal value may not be achievable in the set of stationary policies. In this paper, we study the existence conditions of an optimal stationary policy in a countable-state…
Motivated by applications in movement ecology, in this paper I propose a new class of integrated continuous-time hidden Markov models in which each observation depends on the underlying state of the process over the whole interval since the…
A definition of metastable states applicable to arbitrary finite state Markov processes satisfying detailed balance is discussed. In particular, we identify a crucial condition that distinguishes genuine metastable states from other types…
For a countable-state Markov decision process we introduce an embedding which produces a finite-state Markov decision process. The finite-state embedded process has the same optimal cost, and moreover, it has the same dynamics as the…
This paper concentrates on the reliability of a discrete-time controlled Markov system with finite states and actions, and aims to give an efficient algorithm for obtaining an optimal (control) policy that makes the system have the maximal…
An unconventional approach for optimal stopping under model ambiguity is introduced. Besides ambiguity itself, we take into account how ambiguity-averse an agent is. This inclusion of ambiguity attitude, via an $\alpha$-maxmin nonlinear…
The study of transformations among pure states via Local Operations assisted by Classical Communication (LOCC) plays a central role in entanglement theory. The main emphasis of these investigations is on the deterministic, or probabilistic…
We investigate a novel modeling approach for end-to-end neural network training using hidden Markov models (HMM) where the transition probabilities between hidden states are modeled and learned explicitly. Most contemporary…
The decay of unstable states when several metastable states are available for occupation is investigated using path-integral techniques. Specifically, a method is described which allows the probabilities with which the metastable states are…
We show that maximum entropy (maxent) models can be modeled with certain kinds of HMMs, allowing us to construct maxent models with hidden variables, hidden state sequences, or other characteristics. The models can be trained using the…
We propose a novel, tractable latent state inference scheme for Markov jump processes, for which exact inference is often intractable. Our approach is based on an entropic matching framework that can be embedded into the well-known…
A Markov process is registered. At random moment $\theta$ the distribution of observed sequence changes. Using probability maximizing approach the optimal stopping rule for detecting the change is identified. Some explicit solution is…