Related papers: The compositional construction of Markov processes

We detail a simple procedure (easily convertible to an algorithm) for constructing from quasi-uniform samples of $f$ a sequence of linear spline functions converging to the monotone rearrangement of $f$, in the case where $f$ is an almost…

In this paper, we utilize the framework of Markov processes to attain a more probabilistic perspective on the theory of transfer operators. In doing so, we establish a functional central limit theorem (FLCT) for an $O(N)$ model associated…

We present a~novel approach to the problem of automated theorem proving. Polynomial cost procedures that recognise sentences belonging to a theory are generated on a basis of a set of axioms of the so-called Truncated Predicate Calculus…

Hidden Markov chains are widely applied statistical models of stochastic processes, from fundamental physics and chemistry to finance, health, and artificial intelligence. The hidden Markov processes they generate are notoriously…

We give a general framework for approximations to combinatorial assemblies, especially suitable to the situation where the number $k$ of components is specified, in addition to the overall size $n$. This involves a Poisson process, which,…

We envision a machine capable of solving mathematical problems. Dividing the quantitative reasoning system into two parts: thought processes and cognitive processes, we provide probabilistic descriptions of the architecture.

We study the satisfiability problem of symbolic finite automata and decompose it into the satisfiability problem of the theory of the input characters and the monadic second-order theory of the indices of accepted words. We use our…

This note summarizes the optimization formulations used in the study of Markov decision processes. We consider both the discounted and undiscounted processes under the standard and the entropy-regularized settings. For each setting, we…

Consider an N-dimensional Markov chain obtained from N one-dimensional random walks by Doob h-transform with the q-Vandermonde determinant. We prove that as N becomes large, these Markov chains converge to an infinite-dimensional Feller…

Much algorithmic research in NLP aims to efficiently manipulate rich formal structures. An algorithm designer typically seeks to provide guarantees about their proposed algorithm -- for example, that its running time or space complexity is…

We consider a certain sequence of random walks. The state space of the n-th random walk is the set of all strict partitions of n (that is, partitions without equal parts). We prove that, as n goes to infinity, these random walks converge to…

We present a dynamic programming-based solution to the problem of maximizing the probability of attaining a target set before hitting a cemetery set for a discrete-time Markov control process. Under mild hypotheses we establish that there…

Compositionality is one of the fundamental abilities of the human reasoning process, that allows to decompose a complex problem into simpler elements. Such property is crucial also for neural networks, especially when aiming for a more…

In this paper, the weak convergence of additive functionals of processes with locally independent increments and with Markov switching in the scheme of Poisson approximation is proved. For the relative compactness, a method proposed by R.…

This paper is a survey of applications of the theory of algorithmic randomness to ergodic theory. We establish various degrees of constructivity for asymptotic laws of probability theory. In the framework of the Kolmogorov approach to the…

In this paper we consider the problem of computing the stationary distribution of nearly completely decomposable Markov processes, a well-established area in the classical theory of Markov processes with broad applications in the design,…

We show that various aspects of k-automatic sequences -- such as having an unbordered factor of length n -- are both decidable and effectively enumerable. As a consequence it follows that many related sequences are either k-automatic or…

A number of algorithms have been developed to solve probabilistic inference problems on belief networks. These algorithms can be divided into two main groups: exact techniques which exploit the conditional independence revealed when the…

We prove a complex Ruelle-Perron-Frobenius theorem for Markov shifts over an infinite alphabet, whence extending results by M. Pollicott from the finite to the infinite alphabet setting. As an application we obtain an extension of renewal…

We present a general framework for applying machine-learning algorithms to the verification of Markov decision processes (MDPs). The primary goal of these techniques is to improve performance by avoiding an exhaustive exploration of the…