Related papers: Markov substitute processes : a new model for ling…
Markov decision processes (MDPs) are a standard model for sequential decision-making problems and are widely used across many scientific areas, including formal methods and artificial intelligence (AI). MDPs do, however, come with the…
We provide an introduction to enumerating and constructing invariants of group representations via character methods. The problem is contextualised via two case studies arising from our recent work: entanglement measures, for characterising…
This paper considers a simulation-based estimator for a general class of Markovian processes and explores some strong consistency properties of the estimator. The estimation problem is defined over a continuum of invariant distributions…
In this work we present a modified neural network model which is capable to simulate Markov Chains. We show how to express and train such a network, how to ensure given statistical properties reflected in the training data and we…
We extend the theory of transience to general dynamical systems with no Markov structure assumed. This is linked to the theory of phase transitions. We also provide examples of new kinds of transient behaviour.
This document is an internet supplement to my book "Partially Observed Markov Decision Processes - From Filtering to Controlled Sensing" published by Cambridge University Press in 2016. This internet supplement contains exercises, examples…
The Markov assumption (MA) is fundamental to the empirical validity of reinforcement learning. In this paper, we propose a novel Forward-Backward Learning procedure to test MA in sequential decision making. The proposed test does not assume…
We construct the non-linear Markov process connected with biological model of bacterial genome recombination. The description of invariant measures of this process gives us the solution of one problem in elementary probability theory.
We consider a type of Markov property for set-indexed processes which is satisfied by all processes with independent increments and which allows us to introduce a transition system theory leading to the construction of the process. A…
Markov-modulated L\'evy processes with two different regimes of restarting are studied. These regimes correspond to the completely renewed process and to the process of Markov modulation, accompanied by jumps. We give explicit expressions…
A statistical model for segmentation and word discovery in continuous speech is presented. An incremental unsupervised learning algorithm to infer word boundaries based on this model is described. Results of empirical tests showing that the…
Markov models are widely used to describe processes of stochastic dynamics. Here, we show that Markov models are a natural consequence of the dynamical principle of Maximum Caliber. First, we show that when there are different possible…
The appearance of numbers enumerating alternating sign matrices in stationary states of certain stochastic processes is reviewed. New conjectures concerning nest distribution functions are presented as well as a bijection between certain…
Markov processes on the lattices with arbitrary dimension are omnipresent in statistical mechanics; however their algebraic description is complete only in dimension 1, for which linear algebra provides many tools complementary to the…
We propose a novel, tractable latent state inference scheme for Markov jump processes, for which exact inference is often intractable. Our approach is based on an entropic matching framework that can be embedded into the well-known…
A sequence of random variables is exchangeable if its joint distribution is invariant under variable permutations. We introduce exchangeable variable models (EVMs) as a novel class of probabilistic models whose basic building blocks are…
We consider continuous-time Markov chains on integers which allow transitions to adjacent states only, with alternating rates. We give explicit formulas for probability generating functions, and also for means, variances and state…
When we speak, write or listen, we continuously make predictions based on our knowledge of a language's grammar. Remarkably, children acquire this grammatical knowledge within just a few years, enabling them to understand and generalise to…
Predictive constructions are a powerful way of characterizing the probability law of stochastic processes with certain forms of invariance, such as exchangeability or Markov exchangeability. When de Finetti-like representation theorems are…
We define a class of probabilistic models in terms of an operator algebra of stochastic processes, and a representation for this class in terms of stochastic parameterized grammars. A syntactic specification of a grammar is mapped to…