Related papers: A Linear Programming Approach to Error Bounds for …
Let $X$ be the constrained random walk on ${\mathbb Z}_+^2$ having increments $(1,0)$, $(-1,1)$, $(0,-1)$ with jump probabilities $\lambda(M_k)$, $\mu_1(M_k)$, and $\mu_2(M_k)$ where $M$ is an irreducible aperiodic finite state Markov…
We consider a Markov control model in discrete time with countable both state space and action space. Using the value function of a suitable long-run average reward problem, we study various reachability/controllability problems. First, we…
We propose an approximation for the first return time distribution of random walks on undirected networks. We combine a message-passing solution with a mean-field approximation, to account for the short- and long-term behaviours…
In this paper, we consider a once-reinforced random walk on the half-line, and give the limiting behaviors of all the moments of its range.
In this paper, we provide a novel algorithm for solving planning and learning problems of Markov decision processes. The proposed algorithm follows a policy iteration-type update by using a rank-one approximation of the transition…
We analyze in detail the discrete--time quantum walk on the line by separating the quantum evolution equation into Markovian and interference terms. As a result of this separation, it is possible to show analytically that the quadratic…
This paper considers an optimal impulse control problem of dynamical systems generated by a flow. The performance criteria are total costs over the infinite time horizon. Apart from the main performance to be minimized, there are multiple…
This paper introduces a new algorithm for numerically computing equilibrium (i.e. stationary) distributions for Markov chains and Markov jump processes with either a very large finite state space or a countably infinite state space. The…
This work studies discrete-time discounted Markov decision processes with continuous state and action spaces and addresses the inverse problem of inferring a cost function from observed optimal behavior. We first consider the case in which…
In this paper, we provide a methodology for computing the probability distribution of sojourn times for a wide class of Markov chains. Our methodology consists in writing out linear systems and matrix equations for generating functions…
We analyze a deterministic form of the random walk on the integer line called the {\em liar machine}, similar to the rotor-router model, finding asymptotically tight pointwise and interval discrepancy bounds versus random walk. This…
We consider random walks perturbed at zero which behave like (possibly different) random walks with i.i.d. increments on each half lines and restarts at $0$ whenever they cross that point. We show that the perturbed random walk, after being…
We address the problem of distributing approximation errors in large-scale quantum programs. It has been known for some time that when compiling quantum algorithms for a fault-tolerant architecture, some operations must be approximated as…
The Semi-Markov property of Continuous Time Random Walks (CTRWs) and their limit processes is utilized, and the probability distributions of the bivariate Markov process $(X(t),V(t))$ are calculated: $X(t)$ is a CTRW limit and $V(t)$ a…
In the regime of bounded transportation costs, additive approximations for the optimal transport problem are reduced (rather simply) to relative approximations for positive linear programs, resulting in faster additive approximation…
In order to approximate the exit time of a one-dimensional diffusion process, we propose an algorithm based on a random walk. Such an algorithm was already introduced in both the Brownian context and in the Ornstein-Uhlenbeck context. Here…
We study space-time fluctuations around a characteristic line for a one-dimensional interacting system known as the random average process. The state of this system is a real-valued function on the integers. New values of the function are…
We study the error of reversible Markov chain Monte Carlo methods for approximating the expectation of a function. Explicit error bounds with respect to different norms of the function are proven. By the estimation the well known…
We prove a scaling limit result for random walk on certain random planar maps with its natural time parametrization. In particular, we show that for $\gamma \in (0,2)$, the random walk on the mated-CRT map with parameter $\gamma$ converges…
Approximate linear programming (ALP) is an efficient approach to solving large factored Markov decision processes (MDPs). The main idea of the method is to approximate the optimal value function by a set of basis functions and optimize…