Related papers: Constructing Strong Markov Processes
Intrinsic computation refers to how dynamical systems store, structure, and transform historical and spatial information. By graphing a measure of structural complexity against a measure of randomness, complexity-entropy diagrams display…
Finding the correct encoding for a generic dynamical system's trajectory is a complicated task: the symbolic sequence needs to preserve the invariant properties from the system's trajectory. In theory, the solution to this problem is found…
We study robust Markov decision processes (RMDPs) with non-rectangular uncertainty sets, which capture interdependencies across states unlike traditional rectangular models. While non-rectangular robust policy evaluation is generally…
A large class of non-Markovian quantum processes in open systems can be formulated through time-local master equations which are not in Lindblad form. It is shown that such processes can be embedded in a Markovian dynamics which involves a…
We prove a large deviation principle on path space for a class of discrete time Markov processes whose state space is the intersection of a regular domain $\L\subset \R^d$ with some lattice of spacing $\e$. Transitions from $x$ to $y$ are…
We consider iterated function systems (finite or countable), together with linear and continuous operators on Hilbert spaces, which enable us to construct Markov-type operators. Under suitable conditions, these Markov-type operators have…
We explicitly construct global strict Lyapunov functions for rapidly time-varying nonlinear control systems. The Lyapunov functions we construct are expressed in terms of oftentimes more readily available Lyapunov functions for the limiting…
This paper is concerned with a compositional approach for constructing finite Markov decision processes of interconnected discrete-time stochastic control systems. The proposed approach leverages the interconnection topology and a notion of…
A formula for the transition density of a Markov process defined by an infinite-dimensional stochastic equation is given in terms of the Ornstein--Uhlenbeck bridge and a useful lower estimate on the density is provided. As a consequence,…
In a general setting we solve the following inverse problem: Given a positive operators $R$, acting on measurable functions on a fixed measure space $(X,\mathcal B_X)$, we construct an associated Markov chain. Specifically, starting with a…
A method of constructing Markov chains on finite state spaces is provided. The chain is specified by three constraints: stationarity, dependence and marginal distributions. The generalized Pythagorean theorem in information geometry plays a…
Piecewise Deterministic Markov Processes (PDMPs) are studied in a general framework. First, different constructions are proven to be equivalent. Second, we introduce a coupling between two PDMPs following the same differential flow which…
This survey describes the recent advances in the construction of Markov partitions for nonuniformly hyperbolic systems. One important feature of this development comes from a finer theory of nonuniformly hyperbolic systems, which we also…
We present a formalism to describe slowly decaying systems in the context of finite Markov chains obeying detailed balance. We show that phase space can be partitioned into approximately decoupled regions, in which one may introduce…
A branching process in a Markovian environment consists of an irreducible Markov chain on a set of "environments" together with an offspring distribution for each environment. At each time step the chain transitions to a new random…
We study hyperbolic attractors of some dynamical systems with apriori given countable Markov partitions. Assuming that contraction is stronger than expansion we construct new Markov rectangles such that their crossections by unstable…
Uncertain partially observable Markov decision processes (uPOMDPs) allow the probabilistic transition and observation functions of standard POMDPs to belong to a so-called uncertainty set. Such uncertainty, referred to as epistemic…
With a simple generic approach, we develop a classification that encodes and measures the strength of completeness (or compactness) properties in various types of spaces and ordered structures. The approach also allows us to encode notions…
Complex systems may often be characterized by their hierarchical dynamics. In this paper do we present a method and an operational algorithm that automatically infer this property in a broad range of systems; discrete stochastic processes.…
We introduce the concept of structured synthesis for Markov decision processes where the structure is induced from finitely many pre-specified options for a system configuration. The resulting synthesis problem is in general a nonlinear…