Related papers: Optimized Markov State Models for Metastable Syste…
The hidden Markov model (HMM) provides a powerful framework for inference in time-varying environments, where the underlying state evolves according to a Markov chain. To address the optimal filtering problem in general dynamic settings, we…
Models of biological systems often have many unknown parameters that must be determined in order for model behavior to match experimental observations. Commonly-used methods for parameter estimation that return point estimates of the…
Hybrid systems whose mode dynamics are governed by non-linear ordinary differential equations (ODEs) are often a natural model for biological processes. However such models are difficult to analyze. To address this, we develop a…
The weighted ensemble (WE) simulation strategy provides unbiased sampling of non-equilibrium processes, such as molecular folding or binding, but the extraction of rate constants relies on characterizing steady state behavior.…
Markov parameters play a key role in system identification. There exists many algorithms where these parameters are estimated using least-squares in a first, pre-processing, step, including subspace identification and multi-step…
We consider the Ising model on the hexagonal lattice evolving according to Metropolis dynamics. We study its metastable behavior in the limit of vanishing temperature when the system is immersed in a small external magnetic field. We…
This paper presents new theory and methodology for the Bayesian estimation of overfitted hidden Markov models, with finite state space. The goal is then to achieve posterior emptying of extra states. A prior configuration is constructed…
We introduce the concept of stationary metastable states (SMS's) in the presence of another more stable state. The stationary nature allows us to study SMS's by using a restricted partition function formalism as advocated by Penrose and…
In this paper, we investigate the state estimation problem over multiple Markovian packet drop channels. In this problem setup, a remote estimator receives measurement data transmitted from multiple sensors over individual channels. By the…
We provide a solution to the problem of determining whether a target pure state can be asymptotically prepared using dissipative Markovian dynamics under fixed locality constraints. Beside recovering existing results for a large class of…
In this paper, we develop a Markov decision process (MDP) formulation for the low--temperature metastable Ising model evolving according to Kawasaki dynamics in a finite box of the two--dimensional square lattice. We analyze how an external…
Many control tasks exhibit similar dynamics that can be modeled as having common latent structure. Hidden-Parameter Markov Decision Processes (HiP-MDPs) explicitly model this structure to improve sample efficiency in multi-task settings.…
Despite the advances in probabilistic model checking, the scalability of the verification methods remains limited. In particular, the state space often becomes extremely large when instantiating parameterized Markov decision processes…
We study one-sided and $\alpha$-correct sequential hypothesis testing for data generated by an ergodic Markov chain. The null hypothesis is that the unknown transition matrix belongs to a prescribed set $P$ of stochastic matrices, and the…
A new mechanism for efficiently solving the Markov decision processes (MDPs) is proposed in this paper. We introduce the notion of reachability landscape where we use the Mean First Passage Time (MFPT) as a means to characterize the…
Markov Decision Processes (MDPs) are a popular class of models suitable for solving control decision problems in probabilistic reactive systems. We consider parametric MDPs (pMDPs) that include parameters in some of the transition…
In this article, we review the metastable hierarchy in low-temperature lattice models. In the first part, we state that for any abstract lattice system governed by a Hamiltonian potential and evolving according to a Metropolis-type…
Many large-scale software systems demonstrate metastable failures. In this class of failures, a stressor such as a temporary spike in workload causes the system performance to drop and, subsequently, the system performance continues to…
This paper develops a new approach to the estimation of the degree of boundedness or stability of multidimensional nonlinear systems with time-dependent nonperiodic coefficients-an essential task in various engineering and natural science…
We address stability of a class of Markovian discrete-time stochastic hybrid systems. This class of systems is characterized by the state-space of the system being partitioned into a safe or target set and its exterior, and the dynamics of…