Related papers: A Linear Programming Approach to Error Bounds for …
In this paper, we present a numerical framework for constructing bounds on stationary performance measures of random walks in the positive orthant using the Markov reward approach. These bounds are established in terms of stationary…
A continuous-time random walk in the quarter plane with homogeneous transition rates is considered. Given a non-negative reward function on the state space, we are interested in the expected stationary performance. Since a direct derivation…
We consider a two-node queue modeled as a two-dimensional random walk. In particular, we consider the case that one or both queues have finite buffers. We develop an approximation scheme based on the Markov reward approach to error bounds…
This work deals with the stationary analysis of two-dimensional partially homogeneous nearest-neighbour random walks. Such type of random walks in the quarter plane are characterized by the fact that the one-step transition probabilities…
Nearest neighbor random walks in the quarter plane that are absorbed when reaching the boundary are studied. The cases of positive and zero drift are considered. Absorption probabilities at a given time and at a given site are made…
We present a comparative study of several algorithms for an in-plane random walk with a variable step. The goal is to check the efficiency of the algorithm in the case where the random walk terminates at some boundary. We recently found…
We consider homogeneous random walks in the quarter-plane. The necessary conditions which characterize random walks of which the invariant measure is a sum of geometric terms are provided in [2,3]. Based on these results, we first develop…
An important step in the Markov reward approach to error bounds on stationary performance measures of Markov chains is to bound the bias terms. Affine functions have been successfully used for these bounds for various models, but there are…
In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with…
The author studies the Cramer-Rao type bound by a linear programming approach. By this approach, he found a necessary and sufficient condition that the Cramer-Rao type bound is attained by a random measurement. In a spin 1/2 system, this…
Enumeration of planar lattice walks is a classical topic in combinatorics, at the cross-roads of several domains (e.g., probability, statistical physics, computer science). The aim of this paper is to propose a new approach to obtain some…
This paper is concerned with the development of rigorous approximations to various expectations associated with Markov chains and processes having non-stationary transition probabilities. Such non-stationary models arise naturally in…
We investigate reflected random walks in the quarter plane, with particular emphasis on the time spent along the reflection boundary axes. Assuming the drift of the random walk lies within the cone, the local time converges -- without the…
The treatment of two-dimensional random walks in the quarter plane leads to Markov processes which involve semi-infinite matrices having Toeplitz or block Toeplitz structure plus a low-rank correction. Finding the steady state probability…
Quantum walks are considered to be quantum counterparts of random walks.They show us impressive probability distributions which are different from those of random walks.That fact has been precisely proved in terms of mathematics and some of…
For a homogeneous random walk in the quarter plane with nearest-neighbor transitions, starting from some state $(i_0,j_0)$, we study the event that the walk reaches the vertical axis, before reaching the horizontal axis. We derive an exact…
The author studies the Cram\'{e}r-Rao type bound by a linear programming approach. By this approach, he found a necessary and sufficient condition that the Cram\'{e}r-Rao type bound is attained by a random measurement. In a spin 1/2 system,…
We study a limit behavior of a sequence of Markov processes (or Markov chains) such that their distributions outside of any neighborhood of a "singular" point attract to some probability law. In any neighborhood of this point the behavior…
We establish a linear programming formulation for the solution of joint chance constrained optimal control problems over finite time horizons. The joint chance constraint may represent an invariance, reachability or reach-avoid…
We propose a novel randomized linear programming algorithm for approximating the optimal policy of the discounted Markov decision problem. By leveraging the value-policy duality and binary-tree data structures, the algorithm adaptively…