Related papers: A Tutorial on the Spectral Theory of Markov Chains
Transition Path Theory (TPT) provides a rigorous statistical characterization of the ensemble of trajectories connecting directly, i.e., without detours, two disconnected (sets of) states in a Markov chain, a stochastic process that…
The conventional perspective on Markov chains considers decision problems concerning the probabilities of temporal properties being satisfied by traces of visited states. However, consider the following query made of a stochastic system…
About two dozens of exactly solvable Markov chains on one-dimensional finite and semi-infinite integer lattices are constructed in terms of convolutions of orthogonality measures of the Krawtchouk, Hahn, Meixner, Charlier, $q$-Hahn,…
We formulate some simple conditions under which a Markov chain may be approximated by the solution to a differential equation, with quantifiable error probabilities. The role of a choice of coordinate functions for the Markov chain is…
An asymptotic model for extreme behavior of certain Markov chains is the "tail chain". Generally taking the form of a multiplicative random walk, it is useful in deriving extremal characteristics such as point process limits. We place this…
Recent work incorporating geometric ideas in Markov chain Monte Carlo is reviewed in order to highlight these advances and their possible application in a range of domains beyond Statistics. A full exposition of Markov chains and their use…
In this paper we construct (nonhomogeneous) quantum Markov chains associated with open quantum random walks. The quantum Markov chain, like the classical Markov chain, is a fundamental tool for the investigation of the basic properties such…
Motivated by a model presented by S. Gudder, we study a quantum generalization of Markov chains and discuss the relation between these maps and open quantum random walks, a class of quantum channels described by S. Attal et al. We consider…
Markov Chains with variable length are useful stochastic models for data compression that avoid the curse of dimensionality faced by that full Markov Chains. In this paper we introduce a Variable Length Markov Chain whose transition…
Verification of infinite-state Markov chains is still a challenge despite several fruitful numerical or statistical approaches. For decisive Markov chains, there is a simple numerical algorithm that frames the reachability probability as…
We introduce a new definition of exponential family of Markov chains, and show that many characteristic properties of the usual exponential family of probability distributions are properly extended to Markov chains. The method of…
Inspired by the classical spectral analysis of birth-death chains using orthogonal polynomials, we study an analogous set of constructions in the context of open quantum dynamics and related walks. In such setting, block tridiagonal…
Perturbation analysis of Markov chains provides bounds on the effect that a change in a Markov transition matrix has on the corresponding stationary distribution. This paper compares and analyzes bounds found in the literature for finite…
Modeling and simulating movement of vehicles in established transportation infrastructures, especially in large urban road networks is an important task. It helps with understanding and handling traffic problems, optimizing traffic…
This article serves as an introduction to the study of networks of social systems. First, we introduce the reader to key mathematical tools to study social networks, including mathematical representations of networks and essential…
We study a class of Unitary Quantum Walks on arbitrary graphs, parameterized by a family of scattering matrices. These Scattering Quantum Walks model the discrete dynamics of a system on the edges of the graph, with a scattering process at…
We quantize graphs (networks) which consist of a finite number of bonds and vertices. We show that the spectral statistics of fully connected graphs is well reproduced by random matrix theory. We also define a classical phase space for the…
We propose a sequential Markov chain Monte Carlo (SMCMC) algorithm to sample from a sequence of probability distributions, corresponding to posterior distributions at different times in on-line applications. SMCMC proceeds as in usual MCMC…
This book introduces the mathematical foundations and techniques that lead to the development and analysis of many of the algorithms that are used in machine learning. It starts with an introductory chapter that describes notation used…
Markov branching systems form a fundamental class of stochastic models that are extensively applied in biology, physics, finance, and other domains. These systems are distinguished by their continuous-time evolution and inherent branching…