Related papers: Perturbed Markov Chains and Information Networks
Based on information theory, we present a method to determine an optimal Markov approximation for modelling and prediction from time series data. The method finds a balance between minimal modelling errors by taking as much as possible…
The assumption that data samples are independently identically distributed is the backbone of many learning algorithms. Nevertheless, datasets often exhibit rich structure in practice, and we argue that there exist some unknown order within…
The approximate uniform sampling of graph realizations with a given degree sequence is an everyday task in several social science, computer science, engineering etc. projects. One approach is using Markov chains. The best available current…
In this paper, we present a novel iterative Monte Carlo method for approximating the stationary probability of a single state of a positive recurrent Markov chain. We utilize the characterization that the stationary probability of a state…
We consider the problem of approximating the stationary distribution of an ergodic Markov chain given a set of sampled transitions. Classical simulation-based approaches assume access to the underlying process so that trajectories of…
We develop Markov chain mixing time estimates for a class of Markov chains with restricted transitions. We assume transitions may occur along a cycle of $n$ nodes and on $n^\gamma$ additional edges, where $\gamma < 1$. We find that the…
Many applications in networked control require intermittent access of a controller to a system, as in event-triggered systems or information constrained control applications. Motivated by such applications and extending previous work on…
We study the computation of lower and upper probabilities of hitting a target set of states for imprecise Markov chains, where transition uncertainty is modelled by a convex set of transition matrices. In the precise case, hitting…
We consider a class of discrete time Markov chains with state space [0,1] and the following dynamics. At each time step, first the direction of the next transition is chosen at random with probability depending on the current location. Then…
For many Markov chains of practical interest, the invariant distribution is extremely sensitive to perturbations of some entries of the transition matrix, but insensitive to others; we give an example of such a chain, motivated by a problem…
The paper is largely of a review nature. It considers two main methods used to study stability and obtain appropriate quantitative estimates of perturbations of (inhomogeneous) Markov chains with continuous time and a finite or countable…
An important problem in applied dynamical systems is to compute the external forcing that provokes the largest response of a desired observable quantity. For this, we investigate the perturbation theory of Markov matrices in connection with…
Motivated by applications of distributed linear estimation, distributed control and distributed optimization, we consider the question of designing linear iterative algorithms for computing the average of numbers in a network. Specifically,…
We consider the problem of estimating the transition rate matrix of a continuous-time Markov chain from a finite-duration realisation of this process. We approach this problem in an imprecise probabilistic framework, using a set of prior…
Motivated by robotic surveillance applications, this paper studies the novel problem of maximizing the return time entropy of a Markov chain, subject to a graph topology with travel times and stationary distribution. The return time entropy…
The paper is concerned with approximating the distribution of a sum W of n integer valued random variables Y_i, whose distributions depend on the state of an underlying Markov chain X. The approximation is in terms of a translated Poisson…
We present a Markov-chain analysis of blockwise-stochastic algorithms for solving partially block-separable optimization problems. Our main contributions to the extensive literature on these methods are statements about the Markov operators…
We consider continuous-space, discrete-time Markov chains on $\mathbb{R}^d$, that admit a finite number $N$ of metastable states. Our main motivation for investigating these processes is to analyse random Poincar\'e maps, which describe…
We consider triangular arrays of Markov chains that converge weakly to a diffusion process. Edgeworth type expansions of third order for transition densities are proved. This is done for time horizons that converge to 0. For this purpose we…
The purpose of this paper is to study the time average behavior of Markov chains with transition probabilities being kernels of completely continuous operators, and therefore to provide a sufficient condition for a class of Markov chains…