Related papers: Inferring microscopic kinetics of a Markov process…
For a network of discrete states with a periodically driven Markovian dynamics, we develop an inference scheme for an external observer who has access to some transitions. Based on waiting-time distributions between these transitions, the…
We present an elementary state augmentation method for a class of static risk measure applied to the total cost for both Markov decision processes and stochastic optimal control, such that dynamic programming equations can be derived on the…
We study approximations of evolving probability measures by an interacting particle system. The particle system dynamics is a combination of independent Markov chain moves and importance sampling/resampling steps. Under global regularity…
Many biological systems can be described by finite Markov models. A general method for simplifying master equations is presented that is based on merging adjacent states. The approach preserves the steady-state probability distribution and…
A Markov state model of the dynamics of a protein-like chain immersed in an implicit hard sphere solvent is derived from first principles for a system of monomers that interact via discontinuous potentials designed to account for local…
This paper deals with the optimal stopping problem under partial observation for piecewise-deterministic Markov processes. We first obtain a recursive formulation of the optimal filter process and derive the dynamic programming equation of…
We consider Piecewise Deterministic Markov Processes (PDMPs) with a finite set of discrete states. In the regime of fast jumps between discrete states, we prove a law of large number and a large deviation principle. In the regime of fast…
In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…
We study the problem of synthesizing a policy that maximizes the entropy of a Markov decision process (MDP) subject to a temporal logic constraint. Such a policy minimizes the predictability of the paths it generates, or dually, maximizes…
We estimate the distance in total variation between the law of a finite state Markov process at time t, starting from a given initial measure, and its unique invariant measure. We derive upper bounds for the time to reach the equilibrium.…
We describe an algorithm for computing the maximal invariant set for a Markov chain with linear safety constraints on the distribution over states. We then propose a Markov chain synthesis method that guarantees finite determination of the…
We present and analyse a micro-macro acceleration method for the Monte Carlo simulation of stochastic differential equations with separation between the (fast) time-scale of individual trajectories and the (slow) time-scale of the…
We investigate nonequilibrium steady-state dynamics in both continuous- and discrete-state stochastic processes. Our analysis focuses on planar diffusion dynamics and their coarse-grained approximations by discrete-state Markov chains.…
Based on information theory, we present a method to determine an optimal Markov approximation for modelling and prediction from time series data. The method finds a balance between minimal modelling errors by taking as much as possible…
Empirical force fields employed in molecular dynamics simulations of complex systems can be optimised to reproduce experimentally determined structural and thermodynamic properties. In contrast, experimental knowledge about the rates of…
Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…
Markov jump processes (MJPs) are continuous-time stochastic processes widely used in a variety of applied disciplines. Inference for MJPs typically proceeds via Markov chain Monte Carlo, the state-of-the-art being a uniformization-based…
Motion planning under uncertainty for an autonomous system can be formulated as a Markov Decision Process with a continuous state space. In this paper, we propose a novel solution to this decision-theoretic planning problem that directly…
Molecule- and particle-based simulations provide the tools to test, in microscopic detail, the validity of classical nucleation theory. In this endeavour, determining nucleation mechanisms and rates for phase separation requires an…
We derive exact expressions for the finite-time statistics of extrema (maximum and minimum) of the spatial displacement and the fluctuating entropy flow of biased random walks. Our approach captures key features of extreme events in…