Related papers: Error Breakdown and Sensitivity Analysis of Dynami…
We present an elementary state augmentation method for a class of static risk measure applied to the total cost for both Markov decision processes and stochastic optimal control, such that dynamic programming equations can be derived on the…
Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…
Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task whose outcome's quality depends on both the amount and the diversity of the sampled regions of the state-action space. Yet, many MDPs are…
Motivated by reduction of computational complexity, this work develops sign-error adaptive filtering algorithms for estimating time-varying system parameters. Different from the previous work on sign-error algorithms, the parameters are…
In order to evaluate and compare different recursive filters, simulation is a common tool and numerous simulation models are widely used as 'benchmark'. In the simulation, the continuous time dynamic system is converted into a discrete-time…
The mean first-passage time (MFPT) is one standard measure for the reaction time in thermally activated barrier-crossing processes. While the relationship between MFPTs and phenomenological rate coefficients is known for systems that…
Predictive modeling involving simulation and sensor data at the same time, is a growing challenge in computational science. Even with large-scale finite element models, a mismatch to the sensor data often remains, which can be attributed to…
This paper studies the performative prediction problem which optimizes a stochastic loss function with data distribution that depends on the decision variable. We consider a setting where the agent(s) provides samples adapted to the…
The well-established methodology for the estimation of hidden semi-Markov models (HSMMs) as hidden Markov models (HMMs) with extended state spaces is further developed to incorporate covariate influences across all aspects of the state…
We have investigated temporal fluctuation of superconducting qubits via the time-resolved measurement for an IBM Quantum system. We found that the qubit error rate abruptly changes during specific time intervals. Each high error state…
The simplest, and most common, stochastic model for population processes, including those from biochemistry and cell biology, are continuous time Markov chains. Simulation of such models is often relatively straightforward as there are…
Dynamic state and parameter estimation methods for dynamic security assessment in power systems are becoming increasingly important for system operators. Usually, the data used for this type of applications stems from phasor measurement…
We observe a length-$n$ sample generated by an unknown,stationary ergodic Markov process (\emph{model}) over a finite alphabet $\mathcal{A}$. Given any string $\bf{w}$ of symbols from $\mathcal{A}$ we want estimates of the conditional…
We present a principled approach for estimating the matrix of microscopic rates among states of a Markov process, given only its stationary state population distribution and a single average global kinetic observable. We adapt Maximum…
Interest in equilibrium-based sampling methods has grown with recent advances in computational hardware and Markov state modeling (MSM) methods, yet outstanding questions remain that hinder widespread adoption. Namely, how do sampling…
We undertake a detailed study of the performance of maximum likelihood (ML) estimators of the density matrix of finite-dimensional quantum systems, in order to interrogate generic properties of frequentist quantum state estimation. Existing…
Determining the kinetic bottlenecks that make transitions between metastable states difficult is key to understanding important physical problems like crystallization, chemical reactions, or protein folding. In all these phenomena, the…
We study infinite horizon Markov decision processes (MDPs) with "fast-slow" structure, where some state variables evolve rapidly ("fast states") while others change more gradually ("slow states"). This structure commonly arises in practice…
Switching state-space models (SSSM) are a very popular class of time series models that have found many applications in statistics, econometrics and advanced signal processing. Bayesian inference for these models typically relies on Markov…
Particle Marginal Metropolis-Hastings (PMMH) is a general approach to Bayesian inference when the likelihood is intractable, but can be estimated unbiasedly. Our article develops an efficient PMMH method that scales up better to higher…