Related papers: Open Markov Processes and Reaction Networks

Birth and death Markov processes can model stochastic physical systems from percolation to disease spread and, in particular, wildfires. We introduce and analyze a birth-death-suppression Markov process as a model of controlled culling of…

Understanding the generative mechanism of a natural system is a vital component of the scientific method. Here, we investigate one of the fundamental steps toward this goal by presenting the minimal generator of an arbitrary binary Markov…

Order-preserving couplings are elegant tools for obtaining robust estimates of the time-dependent and stationary distributions of Markov processes that are too complex to be analyzed exactly. The starting point of this paper is to study…

This paper compiles several aspects of the dynamics of stochastic approximation algorithms with Markov iterate-dependent noise when the iterates are not known to be stable beforehand. We achieve the same by extending the lock-in probability…

Markov decision processes (MDPs) are a popular model for decision-making in the presence of uncertainty. The conventional view of MDPs in verification treats them as state transformers with probabilities defined over sequences of states and…

For discrete-time stochastic processes, there is a close connection between return/waiting times and entropy. Such a connection cannot be straightforwardly extended to the continuous-time setting. Contrarily to the discrete-time case one…

We present the non-Markovian generalization of the widely used stochastic Schrodinger equation. Our result allows to describe open quantum systems in terms of stochastic state vectors rather than density operators, without approximation.…

Thermodynamic uncertainty relations have emerged as universal bounds on current fluctuations in non-equilibrium systems. Here we derive a new bound for a particular class of run-and-tumble type processes using the mathematical framework of…

We present a formalism to describe slowly decaying systems in the context of finite Markov chains obeying detailed balance. We show that phase space can be partitioned into approximately decoupled regions, in which one may introduce…

We present a general approach for computing the dynamic partition function of a continuous-time Markov process. The Ruelle topological pressure is identified with the large deviation function of a physical observable. We construct for the…

We consider the asymmetric simple exclusion process confined to the nonnegative integers with an open boundary at 0. The point 0 is connected to a reservoir where particles are injected and ejected at prescribed rates subject to the…

Exploiting previous results on Markovian dynamics and fluctuation theorems, we study the consequences of memory effects on single realizations of nonequilibrium processes within an open system approach. The entropy production along single…

The master equation and, more generally, Markov processes are routinely used as models for stochastic processes. They are often justified on the basis of randomization and coarse-graining assumptions. Here instead, we derive n-th order…

A general formalism is introduced to allow the steady state of non-Markovian processes on networks to be reduced to equivalent Markovian processes on the same substrates. The example of an epidemic spreading process is considered in detail,…

In this paper, we propose a compositional approach for the construction of finite abstractions (a.k.a. finite Markov decision processes (MDPs)) for networks of discrete-time stochastic control subsystems that are not necessarily…

This PhD thesis deals with the Markov picture of developed turbulence from the theoretical point of view. The thesis consists of two parts. The first part introduces stochastic thermodynamics, the second part aims at transferring the…

We consider two types of discrete-time Markov chains where the state space is a graded poset and the transitions are taken along the covering relations in the poset. The first type of Markov chain goes only in one direction, either up or…

An $\mathbb{R}^d$-valued Markov process $X^{(x)}_t=(X^{1,x_1}_t,\dots,X^{d,x_d}_t)$, $t\ge0,x\in\mathbb{R}^d$ is said to be multi-self-similar with index $(\alpha_1,\dots,\alpha_d)\in[0,\infty)^d$ if the identity in law…

We study gradient drift-diffusion processes on a probability simplex set with finite state Wasserstein metrics, namely finite state Wasserstein common noises. A fact is that the Kolmogorov transition equation of finite reversible Markov…

We consider a stochastic process which is (a) described by a continuous-time Markov chain on only short time-scales and (b) constrained to conserve a number of hidden quantities on long time-scales. We assume that the transition matrix of…