Related papers: Turnpikes and Random Walk
We derive a quenched invariance principle for random walks in random environments whose transition probabilities are defined in terms of weighted cycles of bounded length. To this end, we adapt the proof for random walks among random…
We consider random walks in dynamic random environments and propose a criterion which, if satisfied, allows to decompose the random walk trajectory into i.i.d. increments, and ultimately to prove limit theorems. The criterion involves the…
Representations based on random walks can exploit discrete data distributions for clustering and classification. We extend such representations from discrete to continuous distributions. Transition probabilities are now calculated using a…
A constructive proof is given to the fact that any ergodic Markov chain can be realized as a random walk subject to a synchronizing road coloring. Redundancy (ratio of extra entropy) in such a realization is also studied.
We propose a model of random walks on weighted graphs where the weights are interval valued, and connect it to reversible imprecise Markov chains. While the theory of imprecise Markov chains is now well established, this is a first attempt…
This paper studies discounted Markov Decision Processes (MDPs) with finite sets of states and actions. Value iteration is one of the major methods for finding optimal policies. For each discount factor, starting from a finite number of…
Donsker's invariance principle is shown to hold for random walks in rough path topology. As application, we obtain Donsker-type weak limit theorems for stochastic integrals and differential equations.
This paper has a two-folded purpose. First, we attempt to outline the development of the turnpike theorems in the the last several decades. Second, we study turnpike theorems in finite-horizon two-person zero-sum Markov games on a general…
Recently, in ["The coin-turning walk and its scaling limit", Electronic Journal of Probability, 25 (2020)], the ``coin-turning walk'' was introduced on ${\mathbb Z}$. It is a non-Markovian process where the steps form a (possibly)…
In this note, we try to analyze and clarify the intriguing interplay between some counting problems related to specific thermalized weighted graphs and random walks consistent with such graphs.
In this paper, we introduce and study different dissipativity notions and different turnpike properties for discrete-time stochastic nonlinear optimal control problems. The proposed stochastic dissipativity notions extend the classic notion…
We study random walk on complex networks with transition probabilities which depend on the current and previously visited nodes. By using an absorbing Markov chain we derive an exact expression for the mean first passage time between pairs…
We study a one-dimensional Markov modulated random walk with jumps. It is assumed that amplitudes of jumps as well as a chosen velocity regime are random and depend on a time spent by the process at a previous state of the underlying Markov…
We consider random walks perturbed at zero which behave like (possibly different) random walks with i.i.d. increments on each half lines and restarts at $0$ whenever they cross that point. We show that the perturbed random walk, after being…
Several well-known results from the random matrix theory, such as Wigner's law and the Marchenko--Pastur law, can be interpreted (and proved) in terms of non-backtracking walks on a certain graph. Orthogonal polynomials with respect to the…
We prove a quenched central limit theorem for random walks with bounded increments in a randomly evolving environment on $\mathbb{Z}^d$. We assume that the transition probabilities of the walk depend not too strongly on the environment and…
A number of papers have examined various aspects of "random random" walks on finite groups; the purpose of this article is to provide a survey of this work and to show, bring together, and discuss some of the arguments and results in this…
We obtain non-Gaussian limit laws for one-dimensional random walk in a random environment assuming that the environment is a function of a stationary Markov process. This is an extension of the work of Kesten, M. Kozlov and Spitzer for…
We establish a connection between policy evaluation in Markov decision processes and PageRank in network analysis. For a fixed policy, we show that the value function of a discounted Markov decision process can be obtained, up to an…
In this note, we give an original convergence result for products of independent random elements of motion group. Then we consider dynamic random walks which are inhomogeneous Markov chains whose transition probability of each step is, in…