Related papers: Towards Markov-State Holography
We analyze the quantum jumps of an atom interacting with a cavity field. The strong atom- field interaction makes the cavity transmission depend on the time dependent atomic state, and we present a Hidden Markov Model description of the…
High-dimensional dynamical systems projected onto a reduced-order model cease to be deterministic and are best described by probability distributions in state space. Their equations of motion map onto an evolution operator with a…
Pathways of diffusion observed in real-world systems often require stochastic processes going beyond first-order Markov models, as implicitly assumed in network theory. In this work, we focus on second-order Markov models, and derive an…
The vast combination of material properties seen in nature are achieved by the complexity of the material microstructure. Advanced characterization and physics based simulation techniques have led to generation of extremely large…
We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include…
Hidden Markov models (HMMs) are probabilistic functions of finite Markov chains, or, put in other words, state space models with finite state space. In this paper, we examine subspace estimation methods for HMMs whose output lies a finite…
It has become common to perform kinetic analysis using approximate Koopman operators that transforms high-dimensional time series of observables into ranked dynamical modes. Key to a practical success of the approach is the identification…
In this work, we present a transition-state optimization protocol based on the Mode-Tracking algorithm [J. Chem. Phys. 118 (2003) 1634]. By calculating only the eigenvector of interest instead of diagonalizing the full Hessian matrix and…
This paper gives a method for computing distributions associated with patterns in the state sequence of a hidden Markov model, conditional on observing all or part of the observation sequence. Probabilities are computed for very general…
Many systems are partially stochastic in nature. We have derived data driven approaches for extracting stochastic state machines (Markov models) directly from observed data. This chapter provides an overview of our approach with numerous…
In this paper, we develop methods of nonlinear filtering and prediction of an unobservable Markov chain with a finite set of states. This Markov chain controls coefficients of AR(p) model. Using observations generated by AR(p) model we have…
We study finite horizon optimal switching problems for hidden Markov chain models under partially observable Poisson processes. The controller possesses a finite range of strategies and attempts to track the state of the unobserved state…
Even simply-defined, finite-state generators produce stochastic processes that require tracking an uncountable infinity of probabilistic features for optimal prediction. For processes generated by hidden Markov chains the consequences are…
In this work, we explore the state-space formulation of a network process to recover, from partial observations, the underlying network topology that drives its dynamics. To do so, we employ subspace techniques borrowed from system…
Qualitative opacity of a secret is a security property, which means that a system trajectory satisfying the secret is observation-equivalent to a trajectory violating the secret. In this paper, we study how to synthesize a control policy…
Industrial processes generate a massive amount of monitoring data that can be exploited to uncover hidden time losses in the system. This can be used to enhance the accuracy of maintenance policies and increase the effectiveness of the…
There is an increase in interest to model driving maneuver patterns via the automatic unsupervised clustering of naturalistic sequential kinematic driving data. The patterns learned are often used in transportation research areas such as…
Many ion channels spontaneously switch between different levels of activity. Although this behaviour known as modal gating has been observed for a long time it is currently not well understood. Despite the fact that appropriately…
Time series subject to change in regime have attracted much interest in domains such as econometry, finance or meteorology. For discrete-valued regimes, some models such as the popular Hidden Markov Chain (HMC) describe time series whose…
We study discrete-time, discrete-state multistate Markov models from the perspective of algebraic statistics. These models are widely studied in event history analysis, and are characterized by the state space, the initial distribution and…