Related papers: Smart Finite State Devices: A Modeling Framework f…
Renewable energy sources such as wind and solar power have a high degree of unpredictability and time-variation, which makes balancing demand and supply challenging. One possible way to address this challenge is to harness the inherent…
Markov Decision Processes (MDPs) have been used to formulate many decision-making problems in science and engineering. The objective is to synthesize the best decision (action selection) policies to maximize expected rewards (or minimize…
In response to the increasing deployment of battery storage systems for cost reduction and grid stress mitigation, this study presents the development of a new real-time Markov decision process model to efficiently schedule battery systems…
A Markov decision process (MDP) framework is adopted to represent ensemble control of devices with cyclic energy consumption patterns, e.g., thermostatically controlled loads. Specifically we utilize and develop the class of MDP models…
Because failures in distribution systems caused by extreme weather events directly result in consumers' outages, this paper proposes a state-based decision-making model with the objective of mitigating loss of load to improve the…
We study a class of sequential decision-making problems with augmented predictions, potentially provided by a machine learning algorithm. In this setting, the decision-maker receives prediction intervals for unknown parameters that become…
Learning a Markov Decision Process (MDP) from a fixed batch of trajectories is a non-trivial task whose outcome's quality depends on both the amount and the diversity of the sampled regions of the state-action space. Yet, many MDPs are…
Active classification, i.e., the sequential decision-making process aimed at data acquisition for classification purposes, arises naturally in many applications, including medical diagnosis, intrusion detection, and object tracking. In this…
Battery-less Internet of Things (IoT) devices rely on ambient energy harvesting and therefore require scheduling policies that jointly account for energy intermittency and hard timing constraints. This challenge is especially acute in…
Smart home appliances can time-shift and curtail their power demand to assist demand side management or allow operation with limited power, as in an off-grid application. This paper proposes a scheduling process to start appliances with…
This paper proposes a slot-based energy storage approach for decision-making in the context of an Off-Grid telecommunication operator. We consider network systems powered by solar panels, where harvest energy is stored in a battery that can…
Markov decision processes (MDP) are useful to model optimisation problems in concurrent systems. To verify MDPs with efficient Monte Carlo techniques requires that their nondeterminism be resolved by a scheduler. Recent work has introduced…
Advances in mobile computing technologies have made it possible to monitor and apply data-driven interventions across complex systems in real time. Markov decision processes (MDPs) are the primary model for sequential decision problems with…
Designing decentralized policies for wireless communication networks is a crucial problem, which has only been partially solved in the literature so far. In this paper, we propose the Decentralized Markov Decision Process (Dec-MDP)…
This paper considers two important problems -- on the supply-side and demand-side respectively and studies both in a unified framework. On the supply side, we study the problem of energy sharing among microgrids with the goal of maximizing…
This paper proposes a decision-making approach for the control of distribution systems with distributed energy resources (DERs) equipped with photovoltaic (PV) units and battery energy storage systems (BESS). The objective is to minimize…
In a Shared Mobility on Demand Service (SMoDS), dynamic pricing plays an important role in the form of an incentive for the decision of the empowered passenger on the ride offer. Strategies for determining the dynamic tariff should be…
Demand response (DR) programs engage distributed demand-side resources, e.g., controllable residential and commercial loads, in providing ancillary services for electric power systems. Ensembles of these resources can help reducing system…
In this work, a novel digital channelizer design is developed through the use of a compact, system-level modeling approach. The model efficiently captures key properties of a digital channelizer system and its time-varying operation. The…
The paper concerns design of control systems for Demand Dispatch to obtain ancillary services to the power grid by harnessing inherent flexibility in many loads. The role of "local intelligence" at the load has been advocated in prior work,…