Related papers: Randomization for Markov chains with applications …
In parallel-server systems with a single stream of arrivals (a.k.a. load balancing), Join-the-Shortest-Queue (JSQ) is a popular routing algorithm. There is extensive literature studying this system in various asymptotic regimes, but all…
In this work, we focus on the stationary analysis of a specific class of continuous time Markov-modulated reflected random walks in the quarter plane with applications in the modelling of two-node Markov-modulated queueing networks with…
Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of an irreducible Markov chain, are developed using perturbations. The derivation of these…
We study the stochastic dynamics of a system of interacting species in a stochastic environment by means of a continuous-time Markov chain with transition rates depending on the state of the environment. Models of gene regulation in systems…
This paper considers the problem of randomized influence maximization over a Markovian graph process: given a fixed set of nodes whose connectivity graph is evolving as a Markov chain, estimate the probability distribution (over this fixed…
Upper bounds are derived on the total variation distance between the invariant distributions of two stochastic matrices differing on a subset W of rows. Such bounds depend on three parameters: the mixing time and the minimal expected…
Most modern communication networks include fast rerouting mechanisms, implemented entirely in the data plane, to quickly recover connectivity after link failures. By relying on local failure information only, these data plane mechanisms…
We consider a wireless system with a small number of delay constrained users and a larger number of users without delay constraints. We develop a scheduling algorithm that reacts to time varying channels and maximizes throughput utility (to…
In this paper we develop the elements of the theory of algorithmic randomness in continuous-time Markov chains (CTMCs). Our main contribution is a rigorous, useful notion of what it means for an individual trajectory of a CTMC to be random.…
An algorithm for estimating quasi-stationary distribution of finite state space Markov chains has been proven in a previous paper. Now this paper proves a similar algorithm that works for general state space Markov chains under very general…
Order-preserving couplings are elegant tools for obtaining robust estimates of the time-dependent and stationary distributions of Markov processes that are too complex to be analyzed exactly. The starting point of this paper is to study…
We design a dynamic rate scheduling policy of Markov type via the solution (a social optimal Nash equilibrium point) to a utility-maximization problem over a randomly evolving capacity set for a class of generalized processor-sharing queues…
We study the diffusive transport of Markovian random walks on arbitrary networks with stochastic resetting to multiple nodes. We deduce analytical expressions for the stationary occupation probability and for the mean and global first…
Jackson queuing networks have a lot of practical applications, mainly in services and technologic devices. For the first case, an example are the healthcare networks and, for the second, the computation and telecommunications networks.…
The state of many physical, biological and socio-technical systems evolves by combining smooth local transitions and abrupt resetting events to a set of reference values. The inclusion of the resetting mechanism not only provides the…
In many applications, for example when computing statistics of fast subsystems in a multiscale setting, we wish to find the stationary distributions of systems of continuous time Markov chains. Here we present a class of models that appears…
Randomized rounding is a technique that was originally used to approximate hard offline discrete optimization problems from a mathematical programming relaxation. Since then it has also been used to approximately solve sequential stochastic…
We propose a novel method to directly learn a stochastic transition operator whose repeated application provides generated samples. Traditional undirected graphical models approach this problem indirectly by learning a Markov chain model…
A stochastic model for a mobile network is studied. Users enter the network, and then perform independent Markovian routes between nodes where they receive service according to the Processor-Sharing policy. Once their service requirement is…
This work considers the method of uniformisation for continuous-time Markov chains in the context of chemical reaction networks. Previous work in the literature has shown that uniformisation can be beneficial in the context of…