Related papers: Markov Chains and Dynamical Systems: The Open Syst…
Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…
We revise fundamental concepts in the dynamics of open quantum systems in the light of modern developments in the field. Our aim is to present a unified approach to the quantum evolution of open systems that incorporates the concepts and…
Understanding and predicting how complex systems respond to external perturbations is a central challenge in nonequilibrium statistical physics. Here we consider continuous-time Markov networks, which we subject to perturbations along a…
We explore the dynamics of information systems. We show that the driving force for information dynamics is determined by both the information landscape and information flux which determines the equilibrium time reversible and the…
This article is concerned with stability analysis and stabilization of randomly switched nonlinear systems. These systems may be regarded as piecewise deterministic stochastic systems: the discrete switches are triggered by a stochastic…
In this paper, we study the distributionally robust joint chance constrained Markov decision process. {Utilizing the logarithmic transformation technique,} we derive its deterministic reformulation with bi-convex terms under the…
Markov chain analysis is a key technique in formal verification. A practical obstacle is that all probabilities in Markov models need to be known. However, system quantities such as failure rates or packet loss ratios, etc. are often not --…
We study the Markovianity of a composite system and its subsystems. We show how the dissipative nature of a subsystem's dynamics can be modified without having to change properties of the composite system environment. By preparing different…
We consider the synthesis of control policies for probabilistic systems, modeled by Markov decision processes, operating in partially known environments with temporal logic specifications. The environment is modeled by a set of Markov…
Symbolic dynamics has proven to be an invaluable tool in analyzing the mechanisms that lead to unpredictability and random behavior in nonlinear dynamical systems. Surprisingly, a discrete partition of continuous state space can produce a…
This paper deals with control of partially observable discrete-time stochastic systems. It introduces and studies Markov Decision Processes with Incomplete Information and with semi-uniform Feller transition probabilities. The important…
We consider long-time behavior of dynamical systems perturbed by a small noise. Under certain conditions, a slow component of such a motion, which is most important for long- time evolution, can be described as a motion on the cone of…
Shared information is a measure of mutual dependence among multiple jointly distributed random variables with finite alphabets. For a Markov chain on a tree with a given joint distribution, we give a new proof of an explicit…
We study the problem of learning the transition matrices of a set of Markov chains from a single stream of observations on each chain. We assume that the Markov chains are ergodic but otherwise unknown. The learner can sample Markov chains…
We propose a random adaptation variant of time-varying distributed averaging dynamics in discrete time. We show that this leads to novel interpretations of fundamental concepts in distributed averaging, opinion dynamics, and distributed…
Questions are posed regarding the influence that the column sums of the transition probabilities of a stochastic matrix (with row sums all one) have on the stationary distribution, the mean first passage times and the Kemeny constant of the…
The study of the physical properties of open quantum systems is at the heart of many present investigations which aim to describe their dynamical evolution, on theoretical ground and through physical realizations. Here we develop a…
We present a class of stochastic processes in which the large deviation functions of time-integrated observables exhibit singularities that relate to dynamical phase transitions of trajectories. These illustrative examples include Brownian…
We develop a recently introduced representation of quantum dynamics based on sampling negative Markov chain processes. By introducing particles and antiparticles, this formalism maps generic quantum dynamics onto a Markov process defined…
We extend the theory of transience to general dynamical systems with no Markov structure assumed. This is linked to the theory of phase transitions. We also provide examples of new kinds of transient behaviour.