Related papers: Markov processes on quasi-random graphs
Motivated by applications in reinforcement learning (RL), we study a nonlinear stochastic approximation (SA) algorithm under Markovian noise, and establish its finite-sample convergence bounds under various stepsizes. Specifically, we show…
The semi-random graph process is an adaptive random graph process in which an online algorithm is initially presented an empty graph on $n$ vertices. In each round, a vertex $u$ is presented to the algorithm independently and uniformly at…
We present a method for solving implicit (factored) Markov decision processes (MDPs) with very large state spaces. We introduce a property of state space partitions which we call epsilon-homogeneity. Intuitively, an epsilon-homogeneous…
The discrete class algorithm presented in this paper is an efficient simulation tool for stochastic processes governed by a reasonably small set of transition rates. The algorithm is presented, its performance compared to prevailing methods…
We study the expected adjacency matrix of a uniformly random multigraph with fixed degree sequence $\mathbf{d} \in \mathbb{Z}_+^n$. This matrix arises in a variety of analyses of networked data sets, including modularity-maximization and…
We study various classes of random processes defined on the regular tree $T_d$ that are invariant under the automorphism group of $T_d$. Most important ones are factor of i.i.d. processes (randomized local algorithms), branching Markov…
A study of time homogeneous, real valued Markov processes with a special property and a non-atomic initial distribution is provided. The new notion of a function of evolution of distribution which determines the dependency between one…
Random walks are a fundamental model in applied mathematics and are a common example of a Markov chain. The limiting stationary distribution of the Markov chain represents the fraction of the time spent in each state during the stochastic…
We analyze the properties of degree-preserving Markov chains based on elementary edge switchings in undirected and directed graphs. We give exact yet simple formulas for the mobility of a graph (the number of possible moves) in terms of its…
Using a single-site mean-field approximation (MFA) and Monte Carlo simulations, we examine Ising-like models on directed regular random graphs. The models are directed-network implementations of the Ising model, Ising model with absorbing…
Large ensembles of stochastically evolving interacting particles describe phenomena in diverse fields including statistical physics, neuroscience, biology, and engineering. In such systems, the infinitesimal evolution of each particle…
We consider random walks on marked simple point processes with symmetric jump rates and unbounded jump range. We prove homogenization properties of the associated Markov generators. As an application, we derive the hydrodynamic limit of the…
We revisit the method of small subgraph conditioning, used to establish that random regular graphs are Hamiltonian a.a.s. We refine this method using new technical machinery for random $d$-regular graphs on $n$ vertices that hold not just…
We study the zero-range process on the complete graph. It is a Markov chain model for a microcanonical ensemble. We prove that the process converges to a fluid limit. The fluid limit rapidly relaxes to the appropriate Gibbs distribution.
A simple model to study cooperation is the two-species symbiotic contact process (2SCP), in which two different species spread on a graph and interact by a reduced death rate if both occupy the same vertex, representing a symbiotic…
We propose a method to approximate continuous-time, continuous-state stochastic processes by a discrete-time Markov chain defined on a nonuniform grid. Our method provides exact moment matching for processes whose first and second moments…
We provide a detailed description of the structure of the transition probabilities and of the hitting distributions of boundary components of a manifold with corners for a degenerate strong Markov process arising in population genetics. The…
We analyse the eigenvectors of the adjacency matrix of a random inhomogeneous graph constructed from a specified degree sequence. We assume that the empirical degree sequence has bounded mean and variance. We show that near the edges of the…
The Hamiltonian Mean-Field model (HMF), an inertial XY ferromagnet with infinite-range interactions, has been extensively studied in the last few years, especially due to its long-lived meta-equilibrium states, which exhibit a series of…
We introduce the concept of an imprecise Markov semigroup \(\mathbf Q\). It is a tool that allows us to represent ambiguity around both the transition probabilities and the invariant measure of a continuous-time Markov process via a…