Related papers: Backward Approximate Dynamic Programming with Hidd…
Dynamic Programming suffers from the curse of dimensionality due to large state and action spaces, a challenge further compounded by uncertainties in the environment. To mitigate these issue, we explore an off-policy based Temporal…
This paper studies the approximation of optimal control policies by quantized (discretized) policies for a very general class of Markov decision processes (MDPs). The problem is motivated by applications in networked control systems,…
The rapid expansion of wind and solar energy leads to an increasing volatility in the electricity generation. Previous studies have shown that storage devices provide an opportunity to balance fluctuations in the power grid. An economical…
Renewable energy projects, such as large offshore wind farms, are critical to achieving low-emission targets set by governments. Stochastic computer models allow us to explore future scenarios to aid decision making whilst considering the…
In this paper, we study the operational problem of connected hydro power reservoirs which involves sequential decision-making in an uncertain and dynamic environment. The problem is traditionally formulated as a stochastic dynamic program…
We consider partially observable Markov decision processes (POMDPs) with a set of target states and positive integer costs associated with every transition. The traditional optimization objective (stochastic shortest path) asks to minimize…
Power systems with high penetration of variable renewable generation are vulnerable to periods with low generation. An alternative to retain high dispatchable generation capacity is electric energy storage that enables utilization of…
We study merchant energy production modeled as a compound switching and timing option. The resulting Markov decision process is intractable. State-of-the-art approximate dynamic programming methods applied to realistic instances of this…
The modelling of modern power markets requires the representation of the following main features: (i) a stochastic dynamic decision process, with uncertainties related to renewable production and fuel costs, among others; and (ii) a…
This paper presents models for renewable energy systems with storage, and considers its optimal operation. We model and simulate wind and solar power production using stochastic differential equations as well as storage of the produced…
Energy storage are strategic participants in electricity markets to arbitrage price differences. Future power system operators must understand and predict strategic storage arbitrage behaviors for market power monitoring and capacity…
Wind power is playing an increasingly important role in electricity markets. However, it's inherent variability and uncertainty cause operational challenges and costs as more operating reserves are needed to maintain system reliability.…
This paper tackles the growing issue of excessive data transmission in networks. With increasing traffic, backhaul links and core networks are under significant traffic, leading to the investigation of caching solutions at edge routers.…
Dual control explicitly addresses the problem of trading off active exploration and exploitation in the optimal control of partially unknown systems. While the problem can be cast in the framework of stochastic dynamic programming, exact…
Among several tasks in Machine Learning, a specially important one is that of inferring the latent variables of a system and their causal relations with the observed behavior. Learning a Hidden Markov Model of given stochastic process is a…
To address the environmental concern and improve the economic efficiency, the wind power is rapidly integrated into smart grids. However, the inherent uncertainty of wind energy raises operational challenges. To ensure the cost-efficient,…
This article presents a constrained policy optimization approach for the optimal control of systems under nonstationary uncertainties. We introduce an assumption that we call Markov embeddability that allows us to cast the stochastic…
We present a stochastic model predictive control (MPC) framework for central heating, ventilation, and air conditioning (HVAC) plants. The framework uses real data to forecast and quantify uncertainty of disturbances affecting the system…
There has been widespread interest in the use of grid-level storage to handle the variability from increasing penetrations of wind and solar energy. This problem setting requires optimizing energy storage and release decisions for anywhere…
We present an online stochastic model predictive control framework for demand charge management for a grid-connected consumer with attached electrical energy storage. The consumer we consider must satisfy an inflexible but stochastic…