Related papers: Stochastic Dynamics II: Finite Random Dynamical Sy…
This article presents several results establishing connections be- tween Markov chains and dynamical systems, from the point of view of open systems in physics. We show how all Markov chains can be understood as the information on one…
Applications of stochastic models often involve the evaluation of steady-state performance, which requires solving a set of balance equations. In most cases of interest, the number of equations is infinite or even uncountable. As a result,…
We investigate piecewise deterministic Markov processes (PDMP), where the deterministic dynamics follows a scalar conservation law and random jumps in the system are characterized by changes in the flux function. We show under which…
This paper provides full classification of dynamics for continuous time Markov chains (CTMCs) on the non-negative integers with polynomial transition rate functions. Such stochastic processes are abundant in applications, in particular in…
We study the entropy production of a system with a finite number of states connected by random transition rates. The stationary entropy production, driven out of equilibrium both by asymmetric transition rates and by an external probability…
We develop a new algorithm for the estimation of rare event probabilities associated with the steady-state of a Markov stochastic process with continuous state space $\mathbb R^d$ and discrete time steps (i.e. a discrete-time $\mathbb…
By examining the deterministic limit of a general $\epsilon$-dependent generator for Markovian dynamics, which includes the continuous Fokker-Planck equations and discrete chemical master equations as two special cases, the intrinsic…
We introduce a general model of stochastically generated matrix product states (MPS) in which the local tensors share a common distribution and form a strictly stationary sequence, without requiring spatial independence. Under natural…
The $\epsilon$-machine is a stochastic process' optimal model -- maximally predictive and minimal in size. It often happens that to optimally predict even simply-defined processes, probabilistic models -- including the $\epsilon$-machine --…
Switched linear systems are time-varying nonlinear systems whose dynamics switch between different modes, where each mode corresponds to different linear dynamics. They arise naturally to model unexpected failures, environment uncertainties…
Switching dynamical systems provide a powerful, interpretable modeling framework for inference in time-series data in, e.g., the natural sciences or engineering applications. Since many areas, such as biology or discrete-event systems, are…
We show that a reversible pumping mechanism operating between two states of a kinetic network can give rise to Poisson transitions between these two states. An external observer, for whom the pumping mechanism is not accessible, will…
Robust Markov decision processes (RMDPs) extend standard Markov decision processes (MDPs) to account for uncertainty in the transition probabilities. RMDPs have an uncertainty set that defines a set of possible transition functions, each of…
We analyze the stochastic thermodynamics of systems with continuous space of states. The evolution equation, the rate of entropy production, and other results are obtained by a continuous time limit of a discrete time formulation. We point…
In this paper, we propose a distributionally robust control synthesis for an agent with stochastic dynamics that interacts with other agents under uncertainties and constraints expressed by signal temporal logic (STL). We formulate the…
Stochastic thermodynamics provides the framework to analyze thermodynamic laws and quantities along individual trajectories of small but fully observable systems. If the observable level fails to capture all relevant degrees of freedom,…
We study the evolution of the probability density of ensembles of iterates of the logistic map that advance towards and finally remain at attractors of representative dynamical regimes. We consider the mirror families of superstable…
We investigate fluctuations in the momentum flux across a surface perpendicular to the velocity gradient in a stationary shear flow maintained by either thermostated deterministic or by stochastic boundary conditions. In the deterministic…
Recursive stochastic algorithms have gained significant attention in the recent past due to data driven applications. Examples include stochastic gradient descent for solving large-scale optimization problems and empirical dynamic…
In this paper, we introduce a data-driven framework for synthesis of provably-correct controllers for general nonlinear switched systems under complex specifications. The focus is on systems with unknown disturbances whose effects on the…