Related papers: The compositional construction of Markov processes

We develop a Perron-Frobenius type theory for products of random quantum channels acting on finite-dimensional matrix algebras sampled from a stationary and ergodic stochastic process, which, in keeping with the literature, we call ergodic…

Bisimulation metric is a robust behavioural semantics for probabilistic processes. Given any SOS specification of probabilistic processes, we provide a method to compute for each operator of the language its respective metric…

The paper formalizes and extends the idea of local structure approximation for cellular automata originally proposed by Gutowitz et. al. We start with a review of the construction of a probability measure on the set of bi-infinite strings…

We introduce a method for analyzing the complexity of natural language processing tasks, and for predicting the difficulty new NLP tasks. Our complexity measures are derived from the Kolmogorov complexity of a class of automata --- {\it…

In this paper we develop the theory of {\it polymorphisms} of measure spaces, which is a generalization of the theory of measure-preserving transformations; we describe the main notions and discuss relations to the theory of Markov…

In hierarchical planning for Markov decision processes (MDPs), temporal abstraction allows planning with macro-actions that take place at different time scale in form of sequential composition. In this paper, we propose a novel approach to…

Uncertainty in perception, actuation, and the environment often require multiple attempts for a robotic task to be successful. We study a class of problems providing (1) low-entropy indicators of terminal success / failure, and (2)…

We study a Markov process constructed from the P\'olya sum process, which yields a kind of spatial version of the Chinese restaurant process, where each 'table' is assigned a 'location'. This construction firstly allows a definition of…

We define a Markov process in a forward population model with backward genealogy given by the $\Lambda$-coalescent. This Markov process, called the fixation line, is related to the block counting process through its hitting times. Two…

We consider a risk model with a counting process whose intensity is a Markovian shot-noise process, to resolve one of the disadvantages of the Cram\'er-Lundberg model, namely the constant jump intensity of the Poisson process. Due to this…

We develop a convergent variational perturbation theory for conditional probability densities of Markov processes. The power of the theory is illustrated by applying it to the diffusion of a particle in an anharmonic potential.

Constructing an accurate system model for formal model verification can be both resource demanding and time-consuming. To alleviate this shortcoming, algorithms have been proposed for automatically learning system models based on observed…

A general method to construct recombinant tree approximations for stochastic volatility models is developed and applied to the Heston model for stock price dynamics. In this application, the resulting approximation is a four tuple Markov…

We propose that a quantum particle in a potential in one space dimension can be described by a probabilistic cellular automaton. While the simple updating rule of the automaton is deterministic, the probabilistic description is introduced…

Congruences for stochastic automata are defined, the correspondin factor automata are constructed and investigated for automata ove analytic spaces. We study the behavior under finite and infinite streams. Congruences consist of multiple…

The Infinite Monkey Theorem states that if one monkey randomly hits the keys in front of a typewriter keyboard during an infinite amount of time, any works written by William Shakespeare will almost surely be typed out at the end of the…

Bayes' rule tells us how to invert a causal process in order to update our beliefs in light of new evidence. If the process is believed to have a complex compositional structure, we may observe that the inversion of the whole can be…

We study the optimization of the expected long-term reward in finite partially observable Markov decision processes over the set of stationary stochastic policies. In the case of deterministic observations, also known as state aggregation,…

The goal of this work is to formally abstract a Markov process evolving in discrete time over a general state space as a finite-state Markov chain, with the objective of precisely approximating its state probability distribution in time,…

We show that for several variations of partially observable Markov decision processes, polynomial-time algorithms for finding control policies are unlikely to or simply don't have guarantees of finding policies within a constant factor or a…