Related papers: Error Breakdown and Sensitivity Analysis of Dynami…
Over the last decade, hidden Markov models (HMMs) have become increasingly popular in statistical ecology, where they constitute natural tools for studying animal behavior based on complex sensor data. Corresponding analyses sometimes…
We study the approximation of a Markov chain on a reduced state space, for both discrete- and continuous-time Markov chains. In this context, we extend the existing theory of formal error bounds for the approximated transient distributions.…
In this paper we study Monte Carlo estimators based on the likelihood ratio approach for steady-state sensitivity. We first extend the result of Glynn and Olvera-Cravioto [doi:doi: 10.1287/stsy.2018.002] to the setting of continuous time…
Markov state models (MSMs)---or discrete-time master equation models---are a powerful way of modeling the structure and function of molecular systems like proteins. Unfortunately, MSMs with sufficiently many states to make a quantitative…
Several techniques have been recently introduced to mitigate errors in near-term quantum computers without the overhead required by quantum error correcting codes. While most of the focus has been on gate errors, measurement errors are…
In many engineering systems, proper predictive maintenance and operational control are essential to increase efficiency and reliability while reducing maintenance costs. However, one of the major challenges is that many sensors are used for…
A networked system often uses a shared communication network to transmit the measurements to a remotely located estimation center. Due to the limited bandwidth of the channel, a delay may appear while receiving the measurements. This delay…
Dynamic and evolving operational and economic environments present significant challenges for decision-making. We explore a simulation optimization problem characterized by non-stationary input distributions with regime-switching dynamics…
Data analysts are essential in organizations, transforming raw data into insights that drive decision-making and strategy. This study explores how analysts' productivity evolves on a collaborative platform, focusing on two key learning…
We consider the problem of learning a policy for a Markov decision process consistent with data captured on the state-actions pairs followed by the policy. We assume that the policy belongs to a class of parameterized policies which are…
Bayesian methods are increasingly being applied to parameterize mechanistic process models used in environmental prediction and forecasting. In particular, models describing ecosystem dynamics with multiple states that are linear and…
State-space mixed-frequency vector autoregressions are now widely used for nowcasting. Despite their popularity, estimating such models can be computationally intensive, especially for large systems with stochastic volatility. To tackle the…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
This paper investigates the optimization problem of an infinite stage discrete time Markov decision process (MDP) with a long-run average metric considering both mean and variance of rewards together. Such performance metric is important…
In many practical settings control decisions must be made under partial/imperfect information about the evolution of a relevant state variable. Partially Observable Markov Decision Processes (POMDPs) is a relatively well-developed framework…
A decision maker records measurements of a finite-state Markov chain corrupted by noise. The goal is to decide when the Markov chain hits a specific target state. The decision maker can choose from a finite set of sampling intervals to pick…
Time series of conformational dynamics in proteins are usually evaluated with hidden Markov models (HMMs). This approach works well if the number of states and their connectivity is known. However, for the multi-domain protein Hsp90, a…
Consider a subject or unit in a longitudinal biomedical, public health, engineering, economic, or social science study which is being monitored over a possibly random duration. Over time this unit experiences competing recurrent events and…
Hidden Markov models (HMMs) are a versatile statistical framework commonly used in ecology to characterize behavioural patterns from animal movement data. In HMMs, the observed data depend on a finite number of underlying hidden states,…
Continuous-time Markov chains are mathematical models that are used to describe the state-evolution of dynamical systems under stochastic uncertainty, and have found widespread applications in various fields. In order to make these models…