Related papers: Stability in Distribution of Randomly Perturbed Qu…
Markov random fields are used to model high dimensional distributions in a number of applied areas. Much recent interest has been devoted to the reconstruction of the dependency structure from independent samples from the Markov random…
The covariance function of a Gauss-Markov process evaluated at points $(s,t)$ admits a representation as a product of a function of $\min(s,t)$ and a function of $\max(s,t)$. We call these functions the covariance factors of a Gauss-Markov…
We define a Markov process on the set of countable graphs with spins. Transitions are local substitutions in the graph. It is proved that the scaling macrodimension is an invariant of such dynamics.
The stability of iterations of affine linear maps $\Psi_{n}(x)=A_{n}x+B_{n}$, $n=1,2,\ldots$, is studied in the presence of a Markovian environment, more precisely, for the situation when $(A_{n},B_{n})_{n\ge 1}$ is modulated by an ergodic…
Let $\Psi_n$ be a product of $n$ independent, identically distributed random matrices $M$, with the properties that $\Psi_n$ is bounded in $n$, and that $M$ has a deterministic (constant) invariant vector. Assuming that the probability of…
Consider a sequence of Markov processes $X^1, X^2,...$ with state space $E$, where $X^N$ has a strong drift to $D \subseteq E$, such that $\Phi(X^N)$ is slow for some appropriate $\Phi: E\to D$. Using the method of martingale problems, we…
A wide class of ``counting'' problems have been studied in Computer Science. Three typical examples are the estimation of - (i) the permanent of an $n\times n$ 0-1 matrix, (ii) the partition function of certain $n-$ particle Statistical…
Markov decision processes continue to gain in popularity for modeling a wide range of applications ranging from analysis of supply chains and queuing networks to cognitive science and control of autonomous vehicles. Nonetheless, they tend…
For piecewise expanding one-dimensional maps without periodic turning points we prove that isolated eigenvalues of small (random) perturbations of these maps are close to isolated eigenvalues of the unperturbed system. (Here ``eigenvalue''…
Finitarily Markovian processes are those processes $\{X_n\}_{n=-\infty}^{\infty}$ for which there is a finite $K$ ($K = K(\{X_n\}_{n=-\infty}^0$) such that the conditional distribution of $X_1$ given the entire past is equal to the…
We study the convergence of random function iterations for finding an invariant measure of the corresponding Markov operator. We call the problem of finding such an invariant measure the stochastic fixed point problem. This generalizes…
The master equation and, more generally, Markov processes are routinely used as models for stochastic processes. They are often justified on the basis of randomization and coarse-graining assumptions. Here instead, we derive n-th order…
We study a family of Markov processes on $\mathcal{P}^{(k)}$, the space of partitions of the natural numbers with at most $k$ blocks. The process can be constructed from a Poisson point process on…
The dynamics in games involving multiple players, who adaptively learn from their past experience, is not yet well understood. We analyzed a class of stochastic games with Markov strategies in which players choose their actions…
Weconsider Markov decision processes arising from a Markov model of an underlying natural phenomenon. Such phenomena are usually periodic (e.g. annual) in time, and so the Markov processes modelling them must be time-inhomogeneous, with…
We introduce a linear-scaling stochastic method to compute real-space maps of any positive local spectral operator in a tight-binding model. By employing positive-definite estimators, the sampling error at each site can be rigorously…
Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…
Linear fractional Galton-Watson branching processes in i.i.d.~random environment are, on the quenched level, intimately connected to random difference equations by the evolution of the random parameters of their linear fractional marginals.…
The paper is devoted to studies of perturbed Markov chains commonly used for description of information networks. In such models, the matrix of transition probabilities for the corresponding Markov chain is usually regularised by adding a…
A simple model of the new notion of "Markov up" processes is proposed; its positive recurrence and ergodic properties are shown under the appropriate conditions.