Related papers: Infinite horizon asymptotic average optimality for…
Restless bandits are an important class of problems with applications in recommender systems, active learning, revenue management and other areas. We consider infinite-horizon discounted restless bandits with many arms where a fixed…
We analyze the infinite horizon minimax average cost Markov Control Model (MCM), for a class of controlled process conditional distributions, which belong to a ball, with respect to total variation distance metric, centered at a known…
We consider the problem of maximizing the expected average reward obtained over an infinite time horizon by $n$ weakly coupled Markov decision processes. Our setup is a substantial generalization of the multi-armed restless bandit problem…
We consider a novel queuing problem where the decision-maker must choose to accept or reject randomly arriving tasks into a no buffer queue which are processed by $N$ identical servers. Each task has a price, which is a positive real…
Consider a system with $K$ parallel queues in which the server for each queue processes jobs at rate $n$ and the total arrival rate to the system is $nK-\upsilon \sqrt{n}$ where $\upsilon \in (0, \infty)$ and $n$ is large. We study…
We study an infinite horizon optimal stopping problem which arises naturally in the optimal timing of a firm/project sale or in the valuation of natural resources: the functional to be maximised is a sum of a discounted running reward and a…
We consider a Markovian single server queue with impatient customers. There is a customer abandonment cost and a holding cost for customers in the system. We consider two versions of the problem. In the first version, customers pay a reward…
A many-server queueing system is considered in which customers arrive according to a renewal process and have service and patience times that are drawn from two independent sequences of independent, identically distributed random variables.…
We consider a class of multi-agent distributed synchronization systems, which are modeled as $n$ particles moving on the real line. This class generalizes the model of a multi-server queueing system, considered in [15], employing so-called…
We consider a dynamic server control problem for two parallel queues with randomly varying connectivity and server switchover time between the queues. At each time slot the server decides either to stay with the current queue or switch to…
We address the multi-agent persistent monitoring problem defined on a set of nodes (targets) interconnected over a network topology. A measure of mean overall node state uncertainty evaluated over a finite period is to be minimized by…
This paper studies load balancing for many-server ($N$ servers) systems. Each server has a buffer of size $b-1,$ and can have at most one job in service and $b-1$ jobs in the buffer. The service time of a job follows the Coxian-2…
This paper is concerned with uniform stabilization and social optimality for general mean field linear quadratic control systems, where subsystems are coupled via individual dynamics and costs, and the state weight is not assumed with the…
Motivated by various distributed control applications, we consider a linear system with Gaussian noise observed by multiple sensors which transmit measurements over a dynamic lossy network. We characterize the stationary optimal sensor…
This work presents a multiscale framework to solve a class of stochastic optimal control problems in the context of robot motion planning and control in a complex environment. In order to handle complications resulting from a large decision…
We consider the infinite-horizon, average-reward restless bandit problem in discrete time. We propose a new class of policies that are designed to drive a progressively larger subset of arms toward the optimal distribution. We show that our…
In cyber-physical systems such as automobiles, measurement data from sensor nodes should be delivered to other consumer nodes such as actuators in a regular fashion. But, in practical systems over unreliable media such as wireless, it is a…
Parallel server systems in transportation, manufacturing, and computing heavily rely on dynamic routing using connected cyber components for computation and communication. Yet, these components remain vulnerable to random malfunctions and…
We consider a discrete-time parallel service system consisting of $n$ heterogeneous single server queues with infinite capacity. Jobs arrive to the system as an i.i.d. process with rate proportional to $n$, and must be immediately…
We study the steady-state performance of parallel-server systems under an immediate routing architecture with two sources of heterogeneity: servers and job classes, subject to compatibility constraints. We focus on the…