Related papers: Correct Approximation of Stationary Distributions
It has been well known for some time that for strictly stationary Markov chains that are ``reversible'', that special symmetry provides special extra features in the mathematical theory. This paper here is primarily a purely expository…
We extend the Longstaff-Schwartz algorithm for approximately solving optimal stopping problems on high-dimensional state spaces. We reformulate the optimal stopping problem for Markov processes in discrete time as a generalized statistical…
We consider the problem of finding the Perron-Frobenius eigenvector of a primitive matrix. Dividing each of the rows of the matrix by the sum of the elements in the row, the resulting new matrix is stochastic. We give a formula for the…
We consider Markovian models on graphs with local dynamics. We show that, under suitable conditions, such Markov chains exhibit both rapid convergence to equilibrium and strong concentration of measure in the stationary distribution. We…
"Quantum trajectories" are solutions of stochastic differential equations also called Belavkin or Stochastic Schr\"odinger Equations. They describe random phenomena in quantum measurement theory. Two types of such equations are usually…
We propose a novel method to directly learn a stochastic transition operator whose repeated application provides generated samples. Traditional undirected graphical models approach this problem indirectly by learning a Markov chain model…
We develop a new bidirectional algorithm for estimating Markov chain multi-step transition probabilities: given a Markov chain, we want to estimate the probability of hitting a given target state in $\ell$ steps after starting from a given…
We study the long time behaviour of a Markov process evolving in $\mathbb{N}$ and conditioned not to hit 0. Assuming that the process comes back quickly from infinity, we prove that the process admits a unique quasi-stationary distribution…
The paper is devoted to studies of perturbed Markov chains commonly used for description of information networks. In such models, the matrix of transition probabilities for the corresponding Markov chain is usually regularised by adding a…
In increasingly digitalized and metered distribution networks, state estimation is generally recognized as a key enabler of advanced network management functionalities. However, despite decades of research, the real-life adoption of state…
Many applications in network analysis require algorithms to sample uniformly at random from the set of all graphs with a prescribed degree sequence. We present a Markov chain based approach which converges to the uniform distribution of all…
We consider the higher-order Markov Chain, and characterize the second order Markov chains admitting every probability distribution vector as a stationary vector. The result is used to construct Markov chains of higher-order with the same…
Markov chains and diffusion processes are indispensable tools in machine learning and statistics that are used for inference, sampling, and modeling. With the growth of large-scale datasets, the computational cost associated with simulating…
We are interested in the connection between a metastable continuous state space Markov process (satisfying e.g. the Langevin or overdamped Langevin equation) and a jump Markov process in a discrete state space. More precisely, we use the…
In this paper, we study a Markov chain-based stochastic gradient algorithm in general Hilbert spaces, aiming at approximating the optimal solution of a quadratic loss function. We establish probabilistic upper bounds on its convergence. We…
We analyze quasi-stationary distributions $\{\mu^{\varepsilon}\}_{\varepsilon>0}$ of a family of Markov chains $\{X^{\varepsilon}\}_{\varepsilon>0}$ that are random perturbations of a bounded, continuous map $F:M\to M$, where $M$ is a…
We introduce a linear-scaling stochastic method to compute real-space maps of any positive local spectral operator in a tight-binding model. By employing positive-definite estimators, the sampling error at each site can be rigorously…
In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with…
The slow processes of metastable stochastic dynamical systems are difficult to access by direct numerical simulation due the sampling problem. Here, we suggest an approach for modeling the slow parts of Markov processes by approximating the…
The paper is concerned with the equilibrium distributions of continuous-time density dependent Markov processes on the integers. These distributions are known typically to be approximately normal, and the approximation error, as measured in…