Related papers: Markov Chains and Dynamical Systems: The Open Syst…
In many dynamical systems in nature, the law of the dynamics changes along with the temporal evolution of the system. These changes are often associated with the occurrence of certain events. The timing of occurrence of these events…
The conventional perspective on Markov chains considers decision problems concerning the probabilities of temporal properties being satisfied by traces of visited states. However, consider the following query made of a stochastic system…
In this paper we propose a model for open Markov chains that can be interpreted as a system of non-interacting particles evolving according to the rules of a Markov chain. The number of particles in the system is not constant, because we…
This project is going to work with one example of stochastic matrix to understand how Markov chains evolve and how to use them to make faster and better decisions only looking to the present state of the system.
The linear response of a dynamical system refers to changes to properties of the system when small external perturbations are applied. We consider the little-studied question of selecting an optimal perturbation so as to (i) maximise the…
High-dimensional dynamical systems projected onto a reduced-order model cease to be deterministic and are best described by probability distributions in state space. Their equations of motion map onto an evolution operator with a…
In this work, we introduce an information-theoretic approach for considering changes in dynamics of finitely dimensional open quantum systems governed by master equations. This experimentally motivated approach arises from considering how…
Population dynamics are often subject to random independent changes in the environment. For the two strategy stochastic replicator dynamic, we assume that stochastic changes in the environment replace the payoffs and variance. This is…
The concepts of probability, statistics and stochastic theory are being successfully used in structural engineering. Markov Chain modelling is a simple stochastic process model that has found its application in both describing stochastic…
In this paper we study various properties of finite stochastic systems or hidden Markov chains as they are alternatively called. We discuss their construction following different approaches and we also derive recursive filtering formulas…
The thermodynamic formalism allows one to access the chaotic properties of equilibrium and out-of-equilibrium systems, by deriving those from a dynamical partition function. The definition that has been given for this partition function…
Stochastic chains represent a wide and key variety of phenomena in many branches of science within the context of Information Theory and Thermodynamics. They are typically approached by a sequence of independent events or by a memoryless…
Understanding and predicting the dynamical properties of systems involving dry friction is a major concern in physics and engineering. It abounds in many mechanical processes, from the sound produced by a violin to the screeching of chalk…
A Markov decision problem is called reversible if the stationary controlled Markov chain is reversible under every stationary Markovian strategy. A natural application in which such problems arise is in the control of Metropolis-Hastings…
In this paper, we outline a model of graph (or network) dynamics based on two ingredients. The first ingredient is a Markov chain on the space of possible graphs. The second ingredient is a semi-Markov counting process of renewal type. The…
Reversible Markov chains play a central role in stochastic modelling and in algorithms such as Markov chain Monte Carlo (MCMC). Motivated by the fundamental importance of reversibility in classical settings, this paper develops a…
Metastability is a physical phenomenon ubiquitous in first order phase transitions. A fruitful mathematical way to approach this phenomenon is the study of rare transitions Markov chains. For Metropolis chains associated with Statistical…
We consider a class of stochastic dynamical systems, called piecewise deterministic Markov processes, with states $(x, \s)\in \O\times \G$, $\O$ being a region in $\bbR^d$ or the $d$--dimensional torus, $\G$ being a finite set. The…
We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators…
In the paper we consider some piecewise deterministic Markov process whose continuous component evolves according to semiflows, which are switched at the jump times of a Poisson process. The associated Markov chain describes the states of…