Related papers: A Compositional Framework for Markov Processes
Here we propose the Donsker-Varadhan-type compactness conditions and prove the joint large deviation principle for the empirical measure and empirical flow of Markov renewal processes (semi-Markov processes) with a countable state space,…
Many natural and engineered systems can be modeled as discrete state Markov processes. Often, only a subset of states are directly observable. Inferring the conditional probability that a system occupies a particular hidden state, given the…
We revisit closed-loop performance guarantees for Model Predictive Control in the deterministic and stochastic cases, which extend to novel performance results applicable to receding horizon control of Partially Observable Markov Decision…
Filtering---estimating the state of a partially observable Markov process from a sequence of observations---is one of the most widely studied problems in control theory, AI, and computational statistics. Exact computation of the posterior…
Complex systems may often be characterized by their hierarchical dynamics. In this paper do we present a method and an operational algorithm that automatically infer this property in a broad range of systems; discrete stochastic processes.…
We formally extend the notion of Markov order to open quantum processes by accounting for the instruments used to probe the system of interest at different times. Our description recovers the classical Markov order property in the…
We study an open discrete-time queueing network that models the collection of data in a multi-hop sensor network. We assume data is generated at the sensor nodes as a discrete-time Bernoulli process. All nodes in the network maintain a…
Monte Carlo simulations of systems of particles such as hard spheres or soft spheres with singular kernels can display around a phase transition prohibitively long convergence times when using traditional Hasting-Metropolis reversible…
The focus of this article is on entropy and Markov processes. We study the properties of functionals which are invariant with respect to monotonic transformations and analyze two invariant "additivity" properties: (i) existence of a…
The main aim of this work is to establish an averaging principle for a wide class of interacting particle systems in the continuum. This principle is an important step in the analysis of Markov evolutions and is usually applied for the…
The paradigm of considering open quantum systems -- i.e. focusing only on the system of interest, and treating the rest of the world as an effective environment -- has proven to be a highly effective way to understand a range of quantum…
Understanding the fluctuations by which phenomenological evolution equations with thermodynamic structure can be enhanced is the key to a general framework of nonequilibrium statistical mechanics. These fluctuations provide an idealized…
Any tripartite state which saturates the strong subadditivity relation for the quantum entropy is defined as the Markov state.A tripartite pure state describing an open system, its environment and their purifying system isa pure Markov…
We present and discuss a general density-matrix description of energy-dissipation and decoherence phenomena in open quantum systems, able to overcome the intrinsic limitations of the conventional Markov approximation. In particular, the…
We present a formalism to describe slowly decaying systems in the context of finite Markov chains obeying detailed balance. We show that phase space can be partitioned into approximately decoupled regions, in which one may introduce…
Markov chains are the de facto finite-state model for stochastic dynamical systems, and Markov decision processes (MDPs) extend Markov chains by incorporating non-deterministic behaviors. Given an MDP and rewards on states, a classical…
Controlling dynamical fluctuations in open quantum systems is essential both for our comprehension of quantum nonequilibrium behaviour and for its possible application in near-term quantum technologies. However, understanding these…
We consider a discrete time hidden Markov model where the signal is a stationary Markov chain. When conditioned on the observations, the signal is a Markov chain in a random environment under the conditional measure. It is shown that this…
Markov branching systems form a fundamental class of stochastic models that are extensively applied in biology, physics, finance, and other domains. These systems are distinguished by their continuous-time evolution and inherent branching…
We seek to infer the parameters of an ergodic Markov process from samples taken independently from the steady state. Our focus is on non-equilibrium processes, where the steady state is not described by the Boltzmann measure, but is…