Related papers: Optimal Asynchronous Dynamic Policies in Energy-Ef…
Interval Markov decision processes are a class of Markov models where the transition probabilities between the states belong to intervals. In this paper, we study the problem of efficient estimation of the optimal policies in Interval…
The models studied in the steady state involve two queues which are served either by a single server whose speed depends on the number of jobs present, or by several parallel servers whose number may be controlled dynamically. Job service…
The combination of electric vehicles (EVs) and renewable energy is taking shape as a potential driver for a future free of fossil fuels. However, the efficient management of the EV fleet is not exempt from challenges. It calls for the…
This letter deals with the optimization problems of stochastic switching buffer networks, where the switching law is governed by Markov process. The dynamical buffer network is introduced, and its application in modeling the car-sharing…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
Efficient virtual machine (VM) management can dramatically reduce energy consumption in data centers. Existing VM management algorithms fall into two categories based on whether the VMs' resource demands are assumed to be static or dynamic.…
This paper studies optimal switching on and o? of the entire service capacity of an M/M/Infinity queue with holding, running and switching costs where the running costs depend only on whether the system is running or not. The goal is to…
We propose an asynchronous iterative scheme that allows a set of interconnected nodes to distributively reach an agreement within a pre-specified bound in a finite number of steps. While this scheme could be adopted in a wide variety of…
Joint pushing and caching is recognized as an efficient remedy to the problem of spectrum scarcity incurred by tremendous mobile data traffic. In this paper, by exploiting storage resources at end users and predictability of user demand…
We design a dynamic rate scheduling policy of Markov type via the solution (a social optimal Nash equilibrium point) to a utility-maximization problem over a randomly evolving capacity set for a class of generalized processor-sharing queues…
Continuous-time Markov decision processes are an important class of models in a wide range of applications, ranging from cyber-physical systems to synthetic biology. A central problem is how to devise a policy to control the system in order…
We consider a small extent sensor network for event detection, in which nodes take samples periodically and then contend over a {\em random access network} to transmit their measurement packets to the fusion center. We consider two…
This paper addresses the joint scheduling problem of stochastic workloads and a hydrogen-enabled distributed energy system in a low-carbon Internet data centers (IDC). Although such workloads can be shifted over temporal and spatial…
We study optimal data pooling for shared learning in two common maintenance operations: condition-based maintenance and spare parts management. We consider a set of systems subject to Poisson input -- the degradation or demand process --…
This work considers the design of online transmission policy in a wireless-powered communication system with a given energy budget. The system design objective is to maximize the long-term throughput of the system exploiting the energy…
The multi-energy management framework of industrial parks advocates energy conversion and scheduling, which takes full advantage of the compensation and temporal availability of multiple energy. However, how to exploit elastic loads and…
Recent studies have shown that power-proportional data centers can save energy cost by dynamically "right-sizing" the data centers based on real-time workload. More servers are activated when the workload increases while some servers can be…
Wireless networks used for Internet of Things (IoT) are expected to largely involve cloud-based computing and processing. Softwarised and centralised signal processing and network switching in the cloud enables flexible network control and…
In the theory of dynamic programming, an optimal policy is a policy whose lifetime value dominates that of all other policies from every possible initial condition in the state space. This raises a natural question: when does optimality…
This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…