Related papers: Markov matrix perturbations to optimize dynamical …
Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy…
Based on information theory, we present a method to determine an optimal Markov approximation for modelling and prediction from time series data. The method finds a balance between minimal modelling errors by taking as much as possible…
The paper is devoted to studies of perturbed Markov chains commonly used for description of information networks. In such models, the matrix of transition probabilities for the corresponding Markov chain is usually regularised by adding a…
This note summarizes the optimization formulations used in the study of Markov decision processes. We consider both the discounted and undiscounted processes under the standard and the entropy-regularized settings. For each setting, we…
In this paper, by using the Brunovsky normal form, we provide a reformulation of the problem consisting in finding the actuator design which minimizes the controllability cost for finite-dimensional linear systems with scalar controls. Such…
In the study of chaotic behaviour of systems of many hard spheres, Lyapunov exponents of small absolute value exhibit interesting characteristics leading to speculations about connections to non-equilibrium statistical mechanics. Analytical…
Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…
Observations which are realizations from some continuous process are frequent in sciences, engineering, economics, and other fields. We consider linear models, with possible random effects, where the responses are random functions in a…
We develop a perturbation theory of quantum (and classical) master equations with slowly varying parameters, applicable to systems which are externally controlled on a time scale much longer than their characteristic relaxation time. We…
Understanding how systems respond to external perturbations is a fundamental challenge in physics, particularly for non-equilibrium and non-stationary processes. The fluctuation-dissipation theorem provides a complete framework for…
Extreme events are ubiquitous in a wide range of dynamical systems, including turbulent fluid flows, nonlinear waves, large scale networks and biological systems. Here, we propose a variational framework for probing conditions that trigger…
Complex network approaches have been successfully applied for studying transport processes in complex systems ranging from road, railway or airline infrastructure over industrial manufacturing to fluid dynamics. Here, we utilize a generic…
Reversibility is a key property of Markov chains, central to algorithms such as Metropolis-Hastings and other MCMC methods. Yet many applications yield non-reversible chains, motivating the problem of approximating them by reversible ones…
We develop a martingale approximation approach to studying the limiting behavior of quadratic forms of Markov chains. We use the technique to examine the asymptotic behavior of lag-window estimators in time series and we apply the results…
Inference, prediction and control of complex dynamical systems from time series is important in many areas, including financial markets, power grid management, climate and weather modeling, or molecular dynamics. The analysis of such highly…
We develop a convergent variational perturbation theory for conditional probability densities of Markov processes. The power of the theory is illustrated by applying it to the diffusion of a particle in an anharmonic potential.
The simulation of complex stochastic network dynamics arising, for instance, from models of coupled biomolecular processes remains computationally challenging. Often, the necessity to scan a models' dynamics over a large parameter space…
We derive various exact results for Markovian systems that spontaneously relax to a non-equilibrium steady-state by using joint probability distributions symmetries of different entropy production decompositions. The analytical approach is…
The basic question in perturbation analysis of Markov chains is: how do small changes in the transition kernels of Markov chains translate to chains in their stationary distributions? Many papers on the subject have shown, roughly, that the…
Systems out of equilibrium, in stationary as well as in nonstationary regimes, display a linear response to energy impulses simply expressed as the sum of two specific temporal correlation functions. There is a natural interpretation of…