Related papers: Binary N-Step Markov Chain as an Exactly Solvable …
Mostof the existing literature on supervised machine learning problems focuses on the case when the training data set is drawn from an i.i.d. sample. However, many practical problems are characterized by temporal dependence and strong…
We consider a Markov chain that iteratively generates a sequence of random finite words in such a way that the $n^{\mathrm{th}}$ word is uniformly distributed over the set of words of length $2n$ in which $n$ letters are $a$ and $n$ letters…
We describe an exact test of the null hypothesis that a Markov chain is nth order versus the alternate hypothesis that it is $(n+1)$-th order. The procedure does not rely on asymptotic properties, but instead builds up the test statistic…
We consider the convergence of a continuous-time Markov chain approximation X^h, h>0, to an R^d-valued Levy process X. The state space of X^h is an equidistant lattice and its Q-matrix is chosen to approximate the generator of X. In…
Perturbation theory for Markov chains addresses the question how small differences in the transitions of Markov chains are reflected in differences between their distributions. We prove powerful and flexible bounds on the distance of the…
In this article we introduce two new perfect simulation algorithms for chains with infinite memory. Both algorithms belong to the coupling of past procedures. The novelty of our approach is that it allows to include unknown states to the…
We consider a Markov chain of point processes such that each state is a super position of an independent cluster process with the previous state as its centre process together with some independent noise process. The model extends earlier…
Knowing when a graphical model is perfect to a distribution is essential in order to relate separation in the graph to conditional independence in the distribution, and this is particularly important when performing inference from data.…
We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we…
We study a Monte Carlo algorithm for simulation of probability distributions based on stochastic step functions, and compare to the traditional Metropolis/Hastings method. Unlike the latter, the step function algorithm can produce an…
Belief Propagation (BP) is a widely used approximation for exact probabilistic inference in graphical models, such as Markov Random Fields (MRFs). In graphs with cycles, however, no exact convergence guarantees for BP are known, in general.…
Cross-sectional observations from a dynamical system can be modeled via steady-state distributions of Markov processes. The major challenge is then to determine whether the process parameters can be identified and estimated from the…
We consider a general method for the approximation of the distribution of a process conditioned to not hit a given set. Existing methods are based on particle system that are failable, in the sense that, in many situations , they are not…
We study the Markov chain on $\mathbf{F}_p$ obtained by applying a function $f$ and adding $\pm\gamma$ with equal probability. When $f$ is a linear function, this is the well-studied Chung--Diaconis--Graham process. We consider two cases:…
Markov chain Monte Carlo (MCMC) algorithms are ubiquitous in probability theory in general and in machine learning in particular. A Markov chain is devised so that its stationary distribution is some probability distribution of interest.…
This work is devoted to the almost sure stabilization of adaptive control systems that involve an unknown Markov chain. The control system displays continuous dynamics represented by differential equations and discrete events given by a…
We define two algorithms for propagating information in classification problems with pairwise relationships. The algorithms are based on contraction maps and are related to non-linear diffusion and random walks on graphs. The approach is…
Sequential or chained models are increasingly prevalent in machine learning for scientific applications, due to their flexibility and ease of development. Chained models are particularly useful when a task is separable into distinct steps…
We develop a practical approach to establish the stability, that is, the recurrence in a given set, of a large class of controlled Markov chains. These processes arise in various areas of applied science and encompass important numerical…
We develop exact Markov chain Monte Carlo methods for discretely-sampled, directly and indirectly observed diffusions. The qualification "exact" refers to the fact that the invariant and limiting distribution of the Markov chains is the…