Related papers: Optimal Asynchronous Dynamic Policies in Energy-Ef…
Designing decentralized policies for wireless communication networks is a crucial problem, which has only been partially solved in the literature so far. In this paper, we propose the Decentralized Markov Decision Process (Dec-MDP)…
We develop a Markovian framework for load balancing that combines classical algorithms such as Power-of-$d$ with auto-scaling mechanisms that allow the net service capacity to scale up or down in response to the current load on the same…
We consider synchronizing properties of Markov decision processes (MDP), viewed as generators of sequences of probability distributions over states. A probability distribution is p-synchronizing if the probability mass is at least p in some…
This paper considers optimal attack attention allocation on remote state estimation in multi-systems. Suppose there are $\mathtt{M}$ independent systems, each of which has a remote sensor monitoring the system and sending its local…
In cyber-physical systems such as automobiles, measurement data from sensor nodes should be delivered to other consumer nodes such as actuators in a regular fashion. But, in practical systems over unreliable media such as wireless, it is a…
Power management in multi-server data centers~especially at scale is a vital issue of increasing importance in cloud computing paradigm. Existing studies mostly consider thresholds on the number of idle servers to switch the servers on or…
We study a dynamic matching setting where homogeneous agents arrive at random according to a Poisson process and randomly form edges yielding a sparse market. Agents stay in the market according to a certain sojourn time and wait to be…
We consider a fundamental pricing model in which a fixed number of units of a reusable resource are used to serve customers. Customers arrive to the system according to a stochastic process and upon arrival decide whether or not to purchase…
Many sequential decision problems can be formulated as Markov Decision Processes (MDPs) where the optimal value function (or cost-to-go function) can be shown to satisfy a monotone structure in some or all of its dimensions. When the state…
Due to densification of wireless networks, there exist abundance of idling computation resources at edge devices. These resources can be scavenged by offloading heavy computation tasks from small IoT devices in proximity, thereby overcoming…
In this paper, we study a continuous-time discounted jump Markov decision process with both controlled actions and observations. The observation is only available for a discrete set of time instances. At each time of observation, one has to…
Motivated by the rapid development of energy harvesting technology and content-aware communication in access networks, this paper considers the push mechanism design in small-cell base stations (SBSs) powered by renewable energy. A user…
For effective integration of building operations into the evolving demand response programs of the power grid, real-time decisions concerning the use of building appliances for grid services must excel on multiple criteria, ranging from the…
We show that combinations of optimal (stationary) policies in unichain Markov decision processes are optimal. That is, let M be a unichain Markov decision process with state space S, action space A and policies \pi_j^*: S -> A (1\leq j\leq…
We study a job-assignment problem in a large-scale server farm system with geographically deployed servers as abstracted computer components (e.g., storage, network links, and processors) that are potentially diverse. We aim to maximize the…
Peer-to-peer (P2P) energy management facilitates decentralized resource allocation among prosumers, improving local hosting capacity for renewables and minimizing energy expenditures while ensuring data privacy through distributed…
The maximum possible throughput (or the rate of job completion) of a multi-server system is typically the sum of the service rates of individual servers. Recent work shows that launching multiple replicas of a job and canceling them as soon…
Owing to large industrial energy consumption, industrial production has brought a huge burden to the grid in terms of renewable energy access and power supply. Due to the coupling of multiple energy sources and the uncertainty of renewable…
The proliferation of ride sharing systems is a major drive in the advancement of autonomous and electric vehicle technologies. This paper considers the joint routing, battery charging, and pricing problem faced by a profit-maximizing…
In this paper, we consider the gradual-impulse control problem of continuous-time Markov decision processes, where the system performance is measured by the expectation of the exponential utility of the total cost. We prove, under very…