相关论文: The compositional construction of Markov processes

Given finitely many consecutive terms of an infinite sequence, we discuss the construction of a polynomial difference equation that the sequence may satisfy. We also present a method to seek a candidate polynomial differential equation for…

We construct a four-parameter family of Markov processes on infinite Gelfand-Tsetlin schemes that preserve the class of central (Gibbs) measures. Any process in the family induces a Feller Markov process on the infinite-dimensional boundary…

Markov chains are a natural and well understood tool for describing one-dimensional patterns in time or space. We show how to infer $k$-th order Markov chains, for arbitrary $k$, from finite data by applying Bayesian methods to both…

We investigate the convergence in distribution of sequential empirical processes of dependent data indexed by a class of functions F. Our technique is suitable for processes that satisfy a multiple mixing condition on a space of functions…

How do cellular automata behave in the limit of a very large number of cells? Is there a continuum limit with simple properties? We attack this problem by mapping certain classes of automata to quantum field theories for which powerful…

When analyzing probabilistic computations, a powerful approach is to first find a martingale---an expression on the program variables whose expectation remains invariant---and then apply the optional stopping theorem in order to infer…

We provide necessary and sufficient conditions for convergence of exponential integrals of Markov additive processes. Other than in the classical L\'evy case studied by Erickson and Maller we have to distinguish between almost sure…

A function on the state space of a Markov chain is a "lumping" if observing only the function values gives a Markov chain. We give very general conditions for lumpings of a large class of algebraically-defined Markov chains, which include…

In this paper we define a class of polynomial functors suited for constructing coalgebras representing processes in which uncertainty plays an important role. In these polynomial functors we include upper and lower probability measures,…

Kolmogorov complexity and algorithmic probability are defined only up to an additive resp. multiplicative constant, since their actual values depend on the choice of the universal reference computer. In this paper, we analyze a natural…

Poissonian ensembles of Markov loops on a finite graph define a random graph process in which the addition of a loop can merge more than two connected components. We study Markov loops on the complete graph derived from a simple random walk…

The paper is a sketch of systematic presentation of distributional limit theorems and their refinements for compound sums. When analyzing, e.g., ergodic semi-Markov systems with discrete or continuous time, this allows us to separate those…

We formulate Bayesian updates in Markov processes by means of path integral techniques and derive the imaginary-time Schr\"{o}dinger equation with likelihood to direct the inference incorporated as a potential for the posterior probability…

Dynamic programming is a class of algorithms used to compute optimal control policies for Markov decision processes. Dynamic programming is ubiquitous in control theory, and is also the foundation of reinforcement learning. In this paper,…

The entropy of a binary symmetric Hidden Markov Process is calculated as an expansion in the noise parameter epsilon. We map the problem onto a one-dimensional Ising model in a large field of random signs and calculate the expansion…

Construct recursively a long string of words w1. .. wn, such that at each step k, w k+1 is a new word with a fixed probability p $\in$ (0, 1), and repeats some preceding word with complementary probability 1 -- p. More precisely, given a…

Weconsider Markov decision processes arising from a Markov model of an underlying natural phenomenon. Such phenomena are usually periodic (e.g. annual) in time, and so the Markov processes modelling them must be time-inhomogeneous, with…

We consider the problem of approximating the probability mass of the set of timed paths under a continuous-time Markov chain (CTMC) that are accepted by a deterministic timed automaton (DTA). As opposed to several existing works on this…

This article presents the complexity of reachability decision problems for parametric Markov decision processes (pMDPs), an extension to Markov decision processes (MDPs) where transitions probabilities are described by polynomials over a…

We consider the problem of finding the Perron-Frobenius eigenvector of a primitive matrix. Dividing each of the rows of the matrix by the sum of the elements in the row, the resulting new matrix is stochastic. We give a formula for the…