Related papers: Stochastic Dynamics II: Finite Random Dynamical Sy…

In stochastic thermodynamics, the entropy production of a thermodynamic system is defined by the irreversibility measured by the logarithm of the ratio of the path probabilities in the forward and reverse processes. We derive the relation…

We consider the continuous-time setting of linear time-invariant (LTI) systems in feedback with multiplicative stochastic uncertainties. The objective of the paper is to characterize the conditions of Mean-Square Stability (MSS) using a…

Loosely speaking, the Shannon entropy rate is used to gauge a stochastic process' intrinsic randomness; the statistical complexity gives the cost of predicting the process. We calculate, for the first time, the entropy rate and statistical…

The infinitesimal transition probability operator for a continuous-time discrete-state Markov process, $\mathcal{Q}$, can be decomposed into a symmetric and a skew-symmetric parts. As recently shown for the case of diffusion processes,…

We propose an expression for the production of entropy for system described by a stochastic dynamics which is appropriate for the case where the reverse transition rate vanishes but the forward transition is nonzero. The expression is…

Under certain conditions, the dynamics of coarse-grained models of solvated proteins can be described using a Markov state model, which tracks the evolution of populations of configurations. The transition rates among states that appear in…

A general formalism is developed to construct a Markov chain model that converges to a one-dimensional map in the infinite population limit. Stochastic fluctuations are therefore internal to the system and not externally specified. For…

We establish concentration inequalities for random dynamical systems (RDSs), assuming that the observables of interest are separately Lipschitz. Under a weak average contraction condition, we obtain deviation bounds for several random…

Markov switching models (MSMs) are probabilistic models that employ multiple sets of parameters to describe different dynamic regimes that a time series may exhibit at different periods of time. The switching mechanism between regimes is…

Stochastic Hybrid Systems (SHS) constitute an important class of mathematical models that integrate discrete stochastic events with continuous dynamics. The time evolution of statistical moments is generally not closed for SHS, in the sense…

Estimating entropy production from real observation data can be difficult due to finite resolution in both space and time and finite measurement statistics. We characterize the statistical error introduced by finite sample size and compare…

We are interested in risk constraints for infinite horizon discrete time Markov decision processes (MDPs). Starting with average reward MDPs, we show that increasing concave stochastic dominance constraints on the empirical distribution of…

We introduce the concept of a Markov influence system (MIS) and analyze its dynamics. An MIS models a random walk in a graph whose edges and transition probabilities change endogenously as a function of the current distribution. This…

In this paper, we consider an integrated MSP-MDP framework which captures features of Markov decision process (MDP) and multistage stochastic programming (MSP). The integrated framework allows one to study a dynamic decision-making process…

A classical approach to formal policy synthesis in stochastic dynamical systems is to construct a finite-state abstraction, often represented as a Markov decision process (MDP). The correctness of these approaches hinges on a behavioural…

In terms of the stochastic process of quantum-mechanical version of Markov chain Monte Carlo method (the MCMC), we analytically derive macroscopically deterministic flow equations of order parameters such as spontaneous magnetization in…

The Mackey-Glass system is a paradigmatic example of a delayed model whose dynamics is particularly complex due to, among other factors, its multistability involving the coexistence of many periodic and chaotic attractors. The prediction of…

The automated synthesis of control policies for stochastic dynamical systems presents significant challenges. A standard approach is to construct a finite-state abstraction of the continuous system, typically represented as a Markov…

We study stochastic delay differential equations (SDDE) where the coefficients depend on the moving averages of the state process. As a first contribution, we provide sufficient conditions under which a linear path functional of the…

Distributed model predictive control methods for uncertain systems often suffer from considerable conservatism and can tolerate only small uncertainties due to the use of robust formulations that are amenable to distributed design and…