Related papers: A solution to the reversible embedding problem for…
We present an algorithm that can efficiently compute a broad class of inferences for discrete-time imprecise Markov chains, a generalised type of Markov chains that allows one to take into account partially specified probabilities and other…
A Markov matrix is embeddable if it can represent a homogeneous continuous-time Markov process. It is well known that if a Markov matrix has real and pairwise-different eigenvalues, then the embeddability can be determined by checking…
Markov chains are fundamental models for stochastic dynamics, with applications in a wide range of areas such as population dynamics, queueing systems, reinforcement learning, and Monte Carlo methods. Estimating the transition matrix and…
This article shows how coupled Markov chains that meet exactly after a random number of iterations can be used to generate unbiased estimators of the solutions of the Poisson equation. Through this connection, we re-derive known unbiased…
In this paper we investigate the continuum limits of a class of Markov chains. The investigation of such limits is motivated by the desire to model very large networks. We show that under some conditions, a sequence of Markov chains…
The stochastic growth-fragmentation model describes the temporal evolution of a structured cell population through a discrete-time and continuous-state Markov chain. The simulations of this stochastic process and its invariant measure are…
Stemmed from the derivation of the optimal control to a stochastic linear-quadratic control problem with Markov jumps, we study one kind of backward stochastic differential equations (BSDEs) that the generator f is affected by a Markovian…
We consider the Markov chain approximations for singular stable-like processes. First we obtain properties of some Markov chains. Then we construct the approximating Markov chains and give a necessary condition for weak convergence of these…
A new approach for optimal estimation of Markov chains with sparse transition matrices is presented.
Kemeny's constant measures the efficiency of a Markov chain in traversing its states. We investigate whether structure-preserving perturbations to the transition probabilities of a reversible Markov chain can improve its connectivity while…
Quantum embedding theories are powerful tools for approximately solving large-scale strongly correlated quantum many-body problems. The main idea of quantum embedding is to glue together a highly accurate quantum theory at the local scale…
A common tool in the practice of Markov Chain Monte Carlo is to use approximating transition kernels to speed up computation when the desired kernel is slow to evaluate or intractable. A limited set of quantitative tools exist to assess the…
A spectral method for identifying lumping in large Markov chains is presented. Identification of meta stable states is treated as a special case. The method is based on spectral analysis of a self-adjoint matrix that is a function of the…
We study the limit behaviour of a generally non-linear ordinary differential equation whose solution is a superadditive generalisation of a stochastic matrix, and provide necessary and sufficient conditions for this solution to be ergodic,…
The paper deals with finite-state Markov decision processes (MDPs) with integer weights assigned to each state-action pair. New algorithms are presented to classify end components according to their limiting behavior with respect to the…
For Markov chains and Markov processes exhibiting a form of stochastic monotonicity (larger states shift up transition probabilities in terms of stochastic dominance), stability and ergodicity results can be obtained using order-theoretic…
This note presents a simple proof of the monotonicity of the invariant distribution of a discrete Markov chain with a finite state space. This answers a question recently raised by David Siegmund.
We consider the matrix completion problem of recovering a structured low rank matrix with partially observed entries with mixed data types. Vast majority of the solutions have proposed computationally feasible estimators with strong…
We consider the entanglement marginal problem, which consists of deciding whether a number of reduced density matrices are compatible with an overall separable quantum state. To tackle this problem, we propose hierarchies of semidefinite…
This paper aims at improving the convergence to equilibrium of finite ergodic Markov chains via permutations and projections. First, we prove that a specific mixture of permuted Markov chains arises naturally as a projection under the KL…