Related papers: Sample Path Moderate Deviation Principle for Queue…
The key assumption underlying linear Markov Decision Processes (MDPs) is that the learner has access to a known feature map $\phi(x, a)$ that maps state-action pairs to $d$-dimensional vectors, and that the rewards and transitions are…
Given devices space $D$, an intensity measure $\lambda m\in(0,\infty)$, a transition kernel $Q$ from the space $D$ to positive real numbers $(0,\infty,$ a path-loss function (which depends on the Euclidean distance between the devices and a…
We analyze the dynamics of moderate fluctuations for macroscopic observables of the random field Curie Weiss model (i.e., standard Curie-Weiss model embedded in a site dependent, i.i.d. random environment). We obtain path space large…
We consider a Markovian load balancing model on a fully-connected network, where calls have Poisson arrivals and exponential durations. The endpoints of each call are uniform over all the links of the network. Each call is routed either…
We consider a dynamic server allocation problem over parallel queues with randomly varying connectivity and server switchover delay between the queues. At each time slot the server decides either to stay with the current queue or switch to…
Stochastic models for performance analysis, optimization and control of queues hinge on a multitude of alternatives for input point processes. In case of bursty traffic, one very popular model is the \textit{Markov Modulated Poisson…
In this paper, we consider a class of continuous-time, continuous-space stochastic optimal control problems. Building upon recent advances in Markov chain approximation methods and sampling-based algorithms for deterministic path planning,…
We provide the first algorithm that under minimal assumptions allows to simulate the stationary waiting-time sequence of a single-server queue backwards in time, jointly with the input processes of the queue (inter-arrival and service…
In this paper, we provide a compositional approach for constructing finite abstractions (a.k.a. finite Markov decision processes (MDPs)) of interconnected discrete-time stochastic switched systems. The proposed framework is based on a…
We study the rate of convergence to equilibrium of the self-repellent random walk and its local time process on the discrete circle $\mathbb{Z}_n$. While the self-repellent random walk alone is non-Markovian since the jump rates depend on…
Motivated by applications that involve setting proper staffing levels for multi-server queueing systems with batch arrivals, we present a thorough study of the queue-length process $\{Q(t); t \geq 0\}$, departure process $\{D(t); t \geq…
We consider a dynamic server control problem for two parallel queues with randomly varying connectivity and server switchover time between the queues. At each time slot the server decides either to stay with the current queue or switch to…
The moderate deviation regime is concerned with the finite block length trade-off between communication cost and error for information processing tasks in the asymptotic regime, where the communication cost approaches a capacity-like…
We study a large-scale patrol problem with state-dependent costs and multi-agent coordination.We consider heterogeneous agents, rather general reward functions, and the capabilities of tracking agents' trajectories.Given the complexity and…
Motivated by the increasing importance of providing delay-guaranteed services in general computing and communication systems, and the recent wide adoption of learning and prediction in network control, in this work, we consider a general…
Let $X$ be the constrained random walk on ${\mathbb Z}_+^2$ having increments $(1,0)$, $(-1,1)$, $(0,-1)$ with jump probabilities $\lambda(M_k)$, $\mu_1(M_k)$, and $\mu_2(M_k)$ where $M$ is an irreducible aperiodic finite state Markov…
In this paper, we study discrete-time absorbing Markov Decision Processes (MDP) with measurable state space and Borel action space with a given initial distribution. For such models, solutions to the characteristic equation that are not…
Demand for studying queueing systems with multiple servers providing correlated services was created about 60 years ago, motivated by various applications. In recent years, the importance of such studies has been significantly increased,…
In this paper, we study averaging principles for a class of time-inhomogeneous stochastic differential equations (SDEs) with slow and fast time-scales, where the drift term in the fast component is time-dependent and only partially…
We consider a stochastic fluid queue served by a constant rate server and driven by a process which is the local time of a certain Markov process. Such a stochastic system can be used as a model in a priority service system, especially when…