Related papers: Response and Sensitivity Using Markov Chains
Perturbation analysis of Markov chains provides bounds on the effect that a change in a Markov transition matrix has on the corresponding stationary distribution. This paper compares and analyzes bounds found in the literature for finite…
In any Markov chain with finite state space the distribution of transition records always belongs to the exponential family. This observation is used to prove a fluctuation theorem, and to show that the dynamical entropy of a stationary…
This paper deals with the analysis of stochastic systems which can be described by a Langevin equation. By the method presented in this paper drift and diffusion terms of the corresponding Fokker-Planck equation can be extracted from the…
Transformers are increasingly adopted for modeling and forecasting time-series, yet their internal mechanisms remain poorly understood from a dynamical systems perspective. In contrast to classical autoregressive and state-space models,…
This paper presents a convex optimization approach to control the density distribution of autonomous mobile agents with two control modes: ON and OFF. The main new characteristic distinguishing this model from standard Markov decision…
Continuous-time Markov chains on non-negative integers can be used for modeling biological systems, population dynamics, and queueing models. Qualitative behaviors of birth-and-death models, typical examples of such one-dimensional…
We introduce and solve a general model of dynamic response under external perturbations. This model captures a wide range of systems out of equilibrium including Ising models of physical systems, social opinions, and population genetics.…
We study general linear transport-reaction systems on an arbitrary dimensional hypercube with periodic boundary conditions. Transport-reaction systems are often used to model the finite speed movement and interaction of particles, bacteria…
We consider optimal control problems for discrete-time random dynamical systems, finding unique perturbations that provoke maximal responses of statistical properties of the system. We treat systems whose transfer operator has an $L^2$…
We consider the problem of estimating parameter sensitivity for Markovian models of reaction networks. Sensitivity values measure the responsiveness of an output to the model parameters. They help in analyzing the network, understanding its…
The work treats systems combining slow and fast motions depending on each other where fast motions are perturbations of families of either dynamical systems or Markov processes with freezed slow variable. In the first case we consider…
Stochastic models in which agents interact with their neighborhood according to a network topology are a powerful modeling framework to study the emergence of complex dynamic patterns in real-world systems. Stochastic simulations are often…
Using a sensitive statistical test we determine whether or not one can detect the breakdown of linear response given observations of deterministic dynamical systems. A goodness-of-fit statistics is developed for a linear statistical model…
We study linear response for families of skew-product dynamical systems with contracting fibres. Our approach is based on a sectional transfer operator acting on families of probability measures along the fibres. The operator allows to…
The dynamics of decisions in complex networks is studied within a Markov process framework using numerical simulations combined with mathematical insight into the process mechanisms. A mathematical discrete-time model is derived based on a…
Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…
By monitoring the sampling of states with different magnetizations in transition matrix procedures a family of accurate and easily implemented techniques are constructed that automatically control the variation of the temperature or energy…
We present an efficient finite difference method for the computation of parameter sensitivities that is applicable to a wide class of continuous time Markov chain models. The estimator for the method is constructed by coupling the perturbed…
Continuous-time Markov chains are used to model stochastic systems where transitions can occur at irregular times, e.g., birth-death processes, chemical reaction networks, population dynamics, and gene regulatory networks. We develop a…
Networked dynamical systems, i.e., systems of dynamical units coupled via nontrivial interaction topologies, constitute models of broad classes of complex systems, ranging from gene regulatory and metabolic circuits in our cells to…