Related papers: Markov Chains and Dynamical Systems: The Open Syst…
The dynamics of dissipative many-body quantum systems sometimes admit an emergent classical description in terms of a Markov chain even though the corresponding state space contains highly entangled states. In particular, a bad-cavity laser…
Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…
Consider an open quantum system with (discrete-time) Markovian dynamics. Our task is to store information in the system in such a way that it can be retrieved perfectly, even after the system is left to evolve for an arbitrarily long time.…
We develop numerical and analytical approaches to calculate mutual information between complete paths of two molecular components embedded into a larger reaction network. In particular, we focus on a continuous-time Markov chain formalism,…
We study a system of $N$ interacting particles on $\bf{Z}$. The stochastic dynamics consists of two components: a free motion of each particle (independent random walks) and a pair-wise interaction between particles. The interaction belongs…
We continue development of the theory of Markov systems initiated in \cite{Wer1}. In this paper, we introduce fundamental Markov systems associated with random dynamical systems and show that the proof of the uniqueness and empiricalness of…
Switching dynamical systems are an expressive model class for the analysis of time-series data. As in many fields within the natural and engineering sciences, the systems under study typically evolve continuously in time, it is natural to…
Stochastic reaction networks are dynamical models of biochemical reaction systems and form a particular class of continuous-time Markov chains on $\mathbb{N}^n$. Here we provide a fundamental characterisation that connects structural…
We consider a stochastic individual based model where each predator searches during a random time and then manipulates its prey or rests. The time distributions may be non-exponential. An age structure allows to describe these interactions…
The study of open quantum systems is important for fundamental issues of quantum physics as well as for technological applications such as quantum information processing. The interaction of a quantum system with it's environment is usually…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
Consider an n-dimensional linear system where it is known that there are at most k<n non-zero components in the initial state. The observability problem, that is the recovery of the initial state, for such a system is considered. We obtain…
We construct a large class of completely positive and trace preserving non-Markovian dynamical maps for an open quantum system. These maps arise from a piecewise dynamics characterized by a continuous time evolution interrupted by jumps,…
We develop the connection between large deviation theory and more applied approaches to stochastic hybrid systems by highlighting a common underlying Hamiltonian structure. A stochastic hybrid system involves the coupling between a…
A novel class of non-reversible Markov chain Monte Carlo schemes relying on continuous-time piecewise-deterministic Markov Processes has recently emerged. In these algorithms, the state of the Markov process evolves according to a…
In this work, we study dynamic programming (DP) algorithms for partially observable Markov decision processes with jointly continuous and discrete state-spaces. We consider a class of stochastic systems which have coupled discrete and…
Realistic quantum mechanical systems are always exposed to an external environment. The presence of the environment often gives rise to a Markovian process in which the system loses information to its surroundings. However, many quantum…
Motivated by the study of the time evolution of random dynamical systems arising in a vast variety of domains --- ranging from physics to ecology ---, we establish conditions for the occurrence of a non-trivial asymptotic behaviour for…
Filtering is concerned with the sequential estimation of the state, and uncertainties, of a Markovian system, given noisy observations. It is particularly difficult to achieve accurate filtering in complex dynamical systems, such as those…
We define the concept of an `open' Markov process, a continuous-time Markov chain equipped with specified boundary states through which probability can flow in and out of the system. External couplings which fix the probabilities of…