Related papers: Inferring microscopic kinetics of a Markov process…

Density tempering (also called density annealing) is a sequential Monte Carlo approach to Bayesian inference for general state models; it is an alternative to Markov chain Monte Carlo. When applied to state space models, it moves a…

Thermalization is one of the most important phenomena in statistical physics. Often, the transition probabilities between different states in the phase space is or can be approximated by constants. In this case, the system can be described…

In MCMC methods, such as the Metropolis-Hastings (MH) algorithm, the Gibbs sampler, or recent adaptive methods, many different strategies can be proposed, often associated in practice to unknown rates of convergence. In this paper we…

Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…

We propose to model the records of the maximum Drawdown in capital markets by means a Piecewise Deterministic Markov Process (PDMP). We derive statistical results such as the mean and variance that describes the sequence of maximum Drawdown…

We propose a numerical method to approximate the value function for the optimal stopping problem of a piecewise deterministic Markov process (PDMP). Our approach is based on quantization of the post jump location---inter-arrival time Markov…

Numerical methods for the description of nonequilibrium many-particle quantum systems such as equation of motion techniques often cannot compute the full statistics of observables but only moments of it, such as mean, variance and…

The efficient calculation of rare-event kinetics in complex dynamical systems, such as the rate and pathways of ligand dissociation from a protein, is a generally unsolved problem. Markov state models can systematically integrate ensembles…

We study continuous-time Markov chains on the non-negative integers under mild regularity conditions (in particular, the set of jump vectors is finite and both forward and backward jumps are possible). Based on the so-called flux balance…

Realistic models of biological processes typically involve interacting components on multiple scales, driven by changing environment and inherent stochasticity. Such models are often analytically and numerically intractable. We revisit a…

Stochastic processes find applications in modelling systems in a variety of disciplines. A large number of stochastic models considered are Markovian in nature. It is often observed that higher order Markov processes can model the data…

We present a simulation methodology for Bayesian estimation of rate parameters in Markov jump processes arising for example in stochastic kinetic models. To handle the problem of missing components and measurement errors in observed data,…

We develop a stochastic approximation-type algorithm to solve finite state/action, infinite-horizon, risk-aware Markov decision processes. Our algorithm has two loops. The inner loop computes the risk by solving a stochastic saddle-point…

We present a general formalism able to derive the kinetic equations of polymer dynamics. It is based on the application of nonequilibrium thermodynamics to analyze the irreversible processes taking place in the conformational space of the…

The evolution of a continuous time Markov process with a finite number of states is usually calculated by the Master equation - a linear differential equations with a singular generator matrix. We derive a general method for reducing the…

Markov state models (MSMs) are widely employed to analyze the kinetics of complex systems. But despite their effectiveness in many applications, MSMs are prone to systematic or statistical errors, often exacerbated by suboptimal…

Applications of stochastic models often involve the evaluation of steady-state performance, which requires solving a set of balance equations. In most cases of interest, the number of equations is infinite or even uncountable. As a result,…

The paper presents efficient approaches for evaluating convergence rate in total variation for finite and general linear Markov chains. The motivation for studying convergence rate in this metric is its usefulness in various limit theorems.…

We estimate a general mixture of Markov jump processes. The key novel feature of the proposed mixture is that the transition intensity matrices of the Markov processes comprising the mixture are entirely unconstrained. The Markov processes…

Markov state models (MSMs) have been broadly adopted for analyzing molecular dynamics trajectories, but the approximate nature of the models that results from coarse-graining into discrete states is a long-known limitation. We show…