Related papers: Vertex-Reinforced Random Walk
A popular perspective in Reinforcement learning (RL) casts the problem as probabilistic inference on a graphical model of the Markov decision process (MDP). The core object of study is the probability of each state-action pair being visited…
In this paper we consider an excited random walk on $\mathbb{Z}$ in identically piled periodic environment. This is a discrete time process on $\mathbb{Z}$ defined by parameters $(p_1,\dots p_M) \in [0,1]^M$ for some positive integer $M$,…
We establish and generalise several bounds for various random walk quantities including the mixing time and the maximum hitting time. Unlike previous analyses, our derivations are based on rather intuitive notions of local expansion…
We consider a particle moving in continuous time as a Markov jump process; its discrete chain is given by an ordinary random walk on ${\mathbb Z}^d$ , and its jump rate at $({\mathbf x},t)$ is given by a fixed function $\varphi$ of the…
A random walk is a basic stochastic process on graphs and a key primitive in the design of distributed algorithms. One of the most important features of random walks is that, under mild conditions, they converge to a stationary distribution…
We study Vertex-Reinforced-Random-Walk on the complete graph with weights of the form $w(n)=n^\alpha$, with $\alpha>1$. Unlike for the Edge-Reinforced-Random-Walk, which in this case localizes a.s. on 2 sites, here we observe various phase…
We consider a discrete-time two-dimensional process $\{(X_{1,n},X_{2,n})\}$ on $\mathbb{Z}^2$ with a background process $\{J_n\}$ on a finite set $S_0$, where individual processes $\{X_{1,n}\}$ and $\{X_{2,n}\}$ are both skip free. We…
We consider a random walk on a multidimensional integer lattice with random bounds on local times, conditioned on the event that it hits a high level before its death. We introduce an auxiliary "core" process that has a regenerative…
In a step reinforced random walk, at each integer time and with a fixed probability p $\in$ (0, 1), the walker repeats one of his previous steps chosen uniformly at random, and with complementary probability 1 -- p, the walker makes an…
We study a random walk on $\mathbb{Z}$ which evolves in a dynamic environment determined by its own trajectory. Sites flip back and forth between two modes, $p$ and $q$. $R$ consecutive right jumps from a site in the $q$-mode are required…
We consider two natural models of random walks on a module $V$ over a finite commutative ring $R$ driven simultaneously by addition of random elements in $V$, and multiplication by random elements in $R$. In the coin-toss walk, either one…
Random walks are the simplest way to explore or search a graph, and have revealed a very useful tool to investigate and characterize the structural properties of complex networks from the real world, e.g. they have been used to identify the…
A step reinforced random walk is a discrete time process with memory such that at each time step, with fixed probability $p \in (0,1)$, it repeats a previously performed step chosen uniformly at random while with complementary probability…
The set of visited sites and the number of visited sites are two basic properties of the random walk trajectory. We consider two independent random walks on a hyper-cubic lattice and study ordering probabilities associated with these…
We consider a discrete-time process adapted to some filtration which lives on a (typically countable) subset of $\mathbb{R}^d$, $d\geq 2$. For this process, we assume that it has uniformly bounded jumps, is uniformly elliptic (can advance…
For a birth-death process subject to catastrophes, defined on the state-space $S=\{r,r+1,r+2,...\}$, with $r$ a positive integer or zero, the first-visit time to a state $k\in S$ is considered and the Laplace transform of its probability…
We study the occupation measure of various sets for a symmetric transient random walk in $Z^d$ with finite variances. Let $\mu^X_n(A)$ denote the occupation time of the set $A$ up to time $n$. It is shown that $\sup_{x\in…
We study a class of Piecewise Deterministic Markov Processes with state space Rd x E where E is a finite set. The continuous component evolves according to a smooth vector field that is switched at the jump times of the discrete coordinate.…
Jim Propp's P-machine, also known as the "rotor router model" is a simple deterministic process that simulates a random walk on a graph. Instead of distributing chips to randomly chosen neighbors, it serves the neighbors in a fixed order.…
We present the first rigorous quantitative analysis of once-reinforced random walks (ORRW) on general graphs, based on a novel change of measure formula.~This enables us to prove large deviations estimates for the range of the walk to have…