Related papers: A linear programming approach to Markov reward err…
We consider Markov decision processes where the state of the chain is only given at chosen observation times and of a cost. Optimal strategies involve the optimisation of observation times as well as the subsequent action values. We…
A random walk is a basic stochastic process on graphs and a key primitive in the design of distributed algorithms. One of the most important features of random walks is that, under mild conditions, they converge to a stationary distribution…
Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…
Let $X$ be the constrained random walk on ${\mathbb Z}_+^2$ having increments $(1,0)$, $(-1,1)$, $(0,-1)$ with jump probabilities $\lambda(M_k)$, $\mu_1(M_k)$, and $\mu_2(M_k)$ where $M$ is an irreducible aperiodic finite state Markov…
We develop a path-based approach to continuous-time random walks on networks with arbitrarily weighted edges. We describe an efficient numerical algorithm for calculating statistical properties of the stochastic path ensemble. After…
Random walks constitute a fundamental mechanism for a large set of dynamics taking place on networks. In this article, we study random walks on weighted networks with an arbitrary degree distribution, where the weight of an edge between two…
The preparation of the stationary distribution of irreducible, time-reversible Markov chains is a fundamental building block in many heuristic approaches to algorithmically hard problems. It has been conjectured that quantum analogs of…
In queuing theory and related problems, it is very important to know the numerical characteristics of an investigated system - both in stationary and non-stationary modes. In some cases, such characteristics can be calculated, but this is…
Optimal designs minimize the number of experimental runs (samples) needed to accurately estimate model parameters, resulting in algorithms that, for instance, efficiently minimize parameter estimate variance. Governed by knowledge of past…
We propose local-biased random walks on general networks where a Markovian walker can choose between different types of biases in each node to define transitions to its neighbors depending on their degrees. For this ergodic dynamics, we…
The rotor-router model is a deterministic process analogous to a simple random walk on a graph. This paper is concerned with a generalized model, functional-router model, which imitates a Markov chain possibly containing irrational…
We introduce a non-equilibrium discrete-time random walk model on multiplex networks, in which at each time step the walker first undergoes a random jump between neighboring nodes in the same layer, and then tries to hop from one node to…
We consider a Markov jump process on a general state space to which we apply a time-dependent weak perturbation over a finite time interval. By martingale-based stochastic calculus, under a suitable exponential moment bound for the…
Classical linear regression is considered for a case when regression parameters depend on the external random environment. The last is described as a continuous time Markov chain with finite state space. Here the expected sojourn times in…
This paper considers stochastic-constrained stochastic optimization where the stochastic constraint is to satisfy that the expectation of a random function is below a certain threshold. In particular, we study the setting where data samples…
The present article explores the application of randomized control techniques in empirical asset pricing and performance evaluation. It introduces geometric random walks, a class of Markov chain Monte Carlo methods, to construct flexible…
We describe a novel algorithm for rounding packing integer programs based on multidimensional Brownian motion in $\mathbb{R}^n$. Starting from an optimal fractional feasible solution $\bar{x}$, the procedure converges in polynomial time to…
We consider random walks in dynamic random environments and propose a criterion which, if satisfied, allows to decompose the random walk trajectory into i.i.d. increments, and ultimately to prove limit theorems. The criterion involves the…
We study a variable length Markov chain model associated with a group of stationary processes that share the same context tree but each process has potentially different conditional probabilities. We propose a new model selection and…
Brownian motion whose infinitesimal variance changes according to a three-state continuous time Markov Chain is studied. This Markov Chain can be viewed as a telegraph process with one on state and two off states. We first derive the…