Related papers: Optimal Asynchronous Dynamic Policies in Energy-Ef…

We consider a Markovian single server queue with impatient customers. There is a customer abandonment cost and a holding cost for customers in the system. We consider two versions of the problem. In the first version, customers pay a reward…

Calculating optimal policies is known to be computationally difficult for Markov decision processes (MDPs) with Borel state and action spaces. This paper studies finite-state approximations of discrete time Markov decision processes with…

A new class of stochastic processes called independent and periodically identically distributed (i.p.i.d.) processes is defined to capture periodically varying statistical behavior. A novel Bayesian theory is developed for detecting a…

This paper studies convergence properties of optimal values and actions for discounted and average-cost Markov Decision Processes (MDPs) with weakly continuous transition probabilities and applies these properties to the stochastic…

Unexpected loads in Cloud data centers may trigger overloaded situation and performance degradation. To guarantee system performance, cloud computing environment is required to have the ability to handle overloads. The existing approaches,…

We consider discrete-time Markov Decision Processes with Borel state and action spaces and universally measurable policies. For several long-run average cost criteria, we establish the following optimality results: the optimal average cost…

For over a decade, processor design has focused on implementing sophisticated policies for various components of the out-of-order pipeline, including cache replacement and prefetching. The prevailing design philosophy has been to build…

In this work, we aim to obtain the optimal tradeoff between the average delay and the average power consumption in a communication system. In our system, the arrivals occur at each timeslot according to a Bernoulli arrival process and are…

We consider the problem of maximizing the expected average reward obtained over an infinite time horizon by $n$ weakly coupled Markov decision processes. Our setup is a substantial generalization of the multi-armed restless bandit problem…

We consider partially observable Markov decision processes (POMDPs) with a set of target states and positive integer costs associated with every transition. The traditional optimization objective (stochastic shortest path) asks to minimize…

For power grids predominantly featuring large synchronous generators (SGs), there exists a significant body of work bridging optimization and control tasks. A generic workflow in such efforts entails: characterizing the steady state of…

In cyber-physical systems where sensors measure the temporal evolution of a given phenomenon of interest and radio communication takes place over short distances, the energy spent for source acquisition and compression may be comparable…

We consider two-stage tandem queueing systems with one dedicated server in each station and a flexible server that can serve both stations. We assume exponential service times, linear holding costs accrued by jobs present in the system, and…

Markov Decision Processes (MDPs) are a formal framework for modeling and solving sequential decision-making problems. In finite-time horizons such problems are relevant for instance for optimal stopping or specific supply chain problems,…

Flexible loads, i.e. the loads whose power trajectory is not bound to a specific one, constitute a sizable portion of current and future electric demand. This flexibility can be used to improve the performance of the grid, should the right…

We study optimal fidelity selection for a human operator servicing a queue of homogeneous tasks. The agent can service a task with a normal or high fidelity level, where fidelity refers to the degree of exactness and precision while…

In this work, we derive optimal transmission policies in an energy harvesting status update system. The system monitors a stochastic process which can be either in a normal or in an alarm state of operation. We capture the freshness of…

In this article, we investigate a dynamic control problem of a production-inventory system. Here, demands arrive at the production unit according to a Poisson process and are processed in an FCFS manner. The processing time of the…

Data center operators are typically faced with three significant problems when running their data centers, i.e., rising electricity bills, growing carbon footprints and unexpected power outages. To mitigate these issues, running data…

Consider a communication network with a source, a relay and a destination. Each time interval, the source may dynamically choose between a few possible coding schemes, based on the channel state, traffic pattern and its own queue status.…