Related papers: Backward Approximate Dynamic Programming with Hidd…

The most common approaches for solving multistage stochastic programming problems in the research literature have been to either use value functions ("dynamic programming") or scenario trees ("stochastic programming") to approximate the…

This paper formed part of a preliminary research report for a risk consultancy and academic research. Stochastic Programming models provide a powerful paradigm for decision making under uncertainty. In these models the uncertainties are…

There is growing interest in the use of grid-level storage to smooth variations in supply that are likely to arise with increased use of wind and solar energy. Energy arbitrage, the process of buying, storing, and selling electricity to…

Approximate dynamic programming is a popular method for solving large Markov decision processes. This paper describes a new class of approximate dynamic programming (ADP) methods- distributionally robust ADP-that address the curse of…

Large scale electricity storage is set to play an increasingly important role in the management of future energy networks. A major aspect of the economics of such projects is captured in arbitrage, i.e. buying electricity when it is cheap…

The paper investigates stochastic resource allocation problems with scarce, reusable resources and non-preemtive, time-dependent, interconnected tasks. This approach is a natural generalization of several standard resource management…

In this paper, we present long-awaited algorithmic advances toward the efficient construction of near-optimal replenishment policies for a true inventory management classic, the economic warehouse lot scheduling problem. While this paradigm…

The probabilistic characteristics of daily wind speed are not well captured by simple density functions such as Normal or Weibull distribuions as suggested by the existing literature. The unmodeled uncertainties can cause unknown influences…

We propose a new methodology to simulate the discounted penalty applied to a wind-farm operator by violating ramp-rate limitation policies. It is assumed that the operator manages a wind turbine plugged into a battery, which either provides…

In the theory of Partially Observed Markov Decision Processes (POMDPs), existence of optimal policies have in general been established via converting the original partially observed stochastic control problem to a fully observed one on the…

We study linear policy approximations for the risk-conscious operation of an industrial energy system with uncertain wind power, significant and variable electricity demand, and high thermal output, as found in a modern foundry. The system…

Stochastic programming can be applied to consider uncertainties in energy system optimization models for capacity expansion planning. However, these models become increasingly large and time-consuming to solve, even without considering…

We propose a novel randomized linear programming algorithm for approximating the optimal policy of the discounted Markov decision problem. By leveraging the value-policy duality and binary-tree data structures, the algorithm adaptively…

Markov Decision Processes (MDPs) offer a fairly generic and powerful framework to discuss the notion of optimal policies for dynamic systems, in particular when the dynamics are stochastic. However, computing the optimal policy of an MDP…

The Wind Storage Integrated System with Power Smoothing Control (PSC) has emerged as a promising solution to ensure both efficient and reliable wind energy generation. However, existing PSC strategies overlook the intricate interplay and…

We consider approximate dynamic programming for the infinite-horizon stationary $\gamma$-discounted optimal control problem formalized by Markov Decision Processes. While in the exact case it is known that there always exists an optimal…

Short-term forecasting models typically assume the availability of input data (features) when they are deployed and in use. However, equipment failures, disruptions, cyberattacks, may lead to missing features when such models are used…

The increasing interest in renewable energy, particularly in wind, has given rise to the necessity of accurate models for the generation of good synthetic wind speed data. Markov chains are often used with this purpose but better models are…

The prediction of wind speed is very important when dealing with the production of energy through wind turbines. In this paper, we show a new nonparametric model, based on semi-Markov chains, to predict wind speed. Particularly we use an…

Covert planning refers to a class of constrained planning problems where an agent aims to accomplish a task with minimal information leaked to a passive observer to avoid detection. However, existing methods of covert planning often…