Related papers: Stochastic Dynamics II: Finite Random Dynamical Sy…

Entropy production is often interpreted as a proxy for microscopic disorder or environmental roughness in stochastic systems. We test this interpretation using controlled simulations of overdamped stochastic dynamics on curved surfaces in…

For a Markovian dynamics on discrete states, the logarithmic ratio of waiting-time distributions between two successive, instantaneous transitions in forward and backward direction is a measure of time-irreversibility. It thus serves as an…

Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…

Stochastic and soft optimal policies resulting from entropy-regularized Markov decision processes (ER-MDP) are desirable for exploration and imitation learning applications. Motivated by the fact that such policies are sensitive with…

We propose a stochastic dynamics to be associated to a deterministic motion defined by a set of first order differential equation. The transitions that defined the stochastic dynamics are unidirectional and the rates are equal to the…

The stochastic entropy generated during the evolution of a system interacting with an environment may be separated into three components, but only two of these have a non-negative mean. The third component of entropy production is…

We study a class of dynamical systems generated by random substitutions, which contains both intrinsically ergodic systems and instances with several measures of maximal entropy. In this class, we show that the measures of maximal entropy…

Symbolic dynamics has proven to be an invaluable tool in analyzing the mechanisms that lead to unpredictability and random behavior in nonlinear dynamical systems. Surprisingly, a discrete partition of continuous state space can produce a…

We consider a generic class of stochastic particle-based models whose state at an instant in time is described by a set of continuous degrees of freedom (e.g. positions), and the length of this set changes stochastically in time due to…

Entropy, its production, and its change in a dynamical system can be understood from either a fully stochastic dynamic description or from a deterministic dynamics exhibiting chaotic behavior. By taking the former approach based on the…

We study a class of multi-stage stochastic programs, which incorporate modeling features from Markov decision processes (MDPs). This class includes structured MDPs with continuous action and state spaces. We extend policy graphs to include…

The reduced density matrix that characterises the state of an open quantum system is a projection from the full density matrix of the quantum system and its environment, and there are many full density matrices consistent with a given…

This is a short review of the statistical mechanical definition of entropy production for systems composed of a large number of interacting components. Emphasis is on open systems driven away from equilibrium where the entropy production…

We consider a stochastic process which is (a) described by a continuous-time Markov chain on only short time-scales and (b) constrained to conserve a number of hidden quantities on long time-scales. We assume that the transition matrix of…

We consider stochastic discrete event dynamic systems that have time evolution represented with two-dimensional state vectors through a vector equation that is linear in terms of an idempotent semiring. The state transitions are governed by…

Even simply-defined, finite-state generators produce stochastic processes that require tracking an uncountable infinity of probabilistic features for optimal prediction. For processes generated by hidden Markov chains the consequences are…

Leveraging recent work on data-driven methods for constructing a finite state space Markov process from dynamical systems, we address two problems for obtaining further reduced statistical representations. The first problem is to extract…

Markov decision processes (MDP) are a well-established model for sequential decision-making in the presence of probabilities. In robust MDP (RMDP), every action is associated with an uncertainty set of probability distributions, modelling…

Traditional thermodynamic trade-off relations usually apply to quantities that depend linearly on probability distributions. In contrast, many important information-theoretic measures, such as entropies, are nonlinear and therefore…

Hidden Markov chains are widely applied statistical models of stochastic processes, from fundamental physics and chemistry to finance, health, and artificial intelligence. The hidden Markov processes they generate are notoriously…