Related papers: Inferring microscopic kinetics of a Markov process…
A general method to describe stochastic dynamics of Markov processes is suggested. The method aims to solve three related problems. The determination of an optimal coordinate for the description of stochastic dynamics. The reconstruction of…
In this study we apply the maximum entropy principle to derive the properly scaled velocity distribution function of Boltzmann equations for mixtures, which leads to a non-isothermal Maxwell-Stefan diffusion model. We also analyze the…
This tutorial describes recently developed general optimality conditions for Markov Decision Processes that have significant applications to inventory control. In particular, these conditions imply the validity of optimality equations and…
Semi-Markov processes play an important role in the effective description of partially accessible systems in stochastic thermodynamics. They occur, for instance, in coarse-graining procedures such as state lumping and when analyzing waiting…
A Markov state model is a powerful tool that can be used to track the evolution of populations of configurations in an atomistic representation of a protein. For a coarse-grained linear chain model with discontinuous interactions, the…
A multi--state life insurance model is naturally described in terms of the intensity matrix of an underlying (time--inhomogeneous) Markov process which describes the dynamics for the states of an insured person. Between and at transitions,…
Last decade witnesses significant methodological and theoretical advances in estimating large precision matrices. In particular, there are scientific applications such as longitudinal data, meteorology and spectroscopy in which the ordering…
A {\it heuristic} approach is proposed to estimate the average speed of particles during binary encounters by using the macroscopic variables with their extended gradient-type which are the fundamental independent variables in {\it extended…
We collect different examples reflect Bolzman--Jaynes theory of maximum entropy principle. This principle proposed that equillibrium of macrosystem (most probable macrostate of the invariant measure of macrosystem) can be find as a solution…
Fluid approximations have seen great success in approximating the macro-scale behaviour of Markov systems with a large number of discrete states. However, these methods rely on the continuous-time Markov chain (CTMC) having a particular…
A major goal of stochastic thermodynamics is to estimate the inevitable dissipation that accompanies particular observable phenomena in an otherwise not fully accessible system. Quantitative results are often formulated as lower bounds on…
Stochastic kinetic models (SKMs) are increasingly used to account for the inherent stochasticity exhibited by interacting populations of species in areas such as epidemiology, population ecology and systems biology. Species numbers are…
The maximum entropy principle advocates to evaluate events' probabilities using a distribution that maximizes entropy among those that satisfy certain expectations' constraints. Such principle can be generalized for arbitrary decision…
We introduce a gradient-based learning method to automatically adapt Markov chain Monte Carlo (MCMC) proposal distributions to intractable targets. We define a maximum entropy regularised objective function, referred to as generalised speed…
We extend the Longstaff-Schwartz algorithm for approximately solving optimal stopping problems on high-dimensional state spaces. We reformulate the optimal stopping problem for Markov processes in discrete time as a generalized statistical…
To quantify the randomness of Markov trajectories with fixed initial and final states, Ekroot and Cover proposed a closed-form expression for the entropy of trajectories of an irreducible finite state Markov chain. Numerous applications,…
We consider large-scale Markov decision processes (MDPs) with a risk measure of variability in cost, under the risk-aware MDPs paradigm. Previous studies showed that risk-aware MDPs, based on a minimax approach to handling risk, can be…
This paper investigates methods for estimating the optimal stochastic control policy for a Markov Decision Process with unknown transition dynamics and an unknown reward function. This form of model-free reinforcement learning comprises…
Inferential methods can be used to integrate experimental informations and molecular simulations. The maximum entropy principle provides a framework for using equilibrium experimental data and it has been shown that replica-averaged…
Switching dynamical systems are an expressive model class for the analysis of time-series data. As in many fields within the natural and engineering sciences, the systems under study typically evolve continuously in time, it is natural to…