Related papers: Smart Finite State Devices: A Modeling Framework f…
This paper investigates the optimization problem of an infinite stage discrete time Markov decision process (MDP) with a long-run average metric considering both mean and variance of rewards together. Such performance metric is important…
An increasing concern in power systems is how to elicit flexibilities in demand, which leads to nontraditional electricity products for accommodating loads of different flexibility levels. We have proposed Multiple-Arrival Multiple-Deadline…
Faced with the complexities of managing natural gas-dependent power system amid the surge of renewable integration and load unpredictability, this study explores strategies for navigating emergency transitions to costlier secondary fuels.…
Data centers are becoming a major consumer of electricity on the grid, with cooling accounting for about 40\% of that energy. As electricity prices vary throughout the day and year, there is a need for cooling strategies that adapt to these…
In this paper, we investigate the scheduling design of a mobile-edge computing (MEC) system, where the random arrival of mobile devices with computation tasks in both spatial and temporal domains is considered. The binary computation…
In order to efficiently provide demand side management (DSM) in smart grid, carrying out pricing on the basis of real-time energy usage is considered to be the most vital tool because it is directly linked with the finances associated with…
Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and…
In this paper, we propose and study the effectiveness of customer engagement plans that clearly specify the amount of intervention in customer's load settings by the grid operator for peak load reduction. We suggest two different types of…
We consider finite model approximations of discrete-time partially observed Markov decision processes (POMDPs) under the discounted cost criterion. After converting the original partially observed stochastic control problem to a fully…
The standard Markov Decision Process (MDP) formulation hinges on the assumption that an action is executed immediately after it was chosen. However, assuming it is often unrealistic and can lead to catastrophic failures in applications such…
This paper presents two new approaches to decomposing and solving large Markov decision problems (MDPs), a partial decoupling method and a complete decoupling method. In these approaches, a large, stochastic decision problem is divided into…
We consider the problem of solving robust Markov decision process (MDP), which involves a set of discounted, finite state, finite action space MDPs with uncertain transition kernels. The goal of planning is to find a robust policy that…
We study the relation between different Markov Decision Process (MDP) frameworks in the machine learning and econometrics literatures, including the standard MDP, the entropy and general regularized MDP, and stochastic MDP, where the latter…
Stochastic and soft optimal policies resulting from entropy-regularized Markov decision processes (ER-MDP) are desirable for exploration and imitation learning applications. Motivated by the fact that such policies are sensitive with…
In the past extensive researches have been conducted on demand side management (DSM) program which aims at reducing peak loads and saving electricity cost. In this paper, we propose a framework to study decentralized household demand side…
Recently, we have struck the balance between the information freshness, in terms of age of information (AoI), experienced by users and energy consumed by sensors, by appropriately activating sensors to update their current status in caching…
Demand side management (DSM) is a key solution for reducing the peak-time power consumption in smart grids. To provide incentives for consumers to shift their consumption to off-peak times, the utility company charges consumers differential…
In the paper, we consider delay-optimal charging scheduling of the electric vehicles (EVs) at a charging station with multiple charge points. The charging station is equipped with renewable energy generation devices and can also buy energy…
In this paper, we propose a novel censoring policy for energy-efficient transmissions in energy-harvesting sensors. The problem is formulated as an infinite-horizon Markov Decision Process (MDP). The objective to be optimized is the…
Markov Decision Processes (MDPs) are a mathematical framework for modeling sequential decision making under uncertainty. The classical approaches for solving MDPs are well known and have been widely studied, some of which rely on…