Related papers: Backward Approximate Dynamic Programming with Hidd…
We consider a sequential decision making process, such as renewable energy trading or electrical production scheduling, whose outcome depends on the future realization of a random factor, such as a meteorological variable. We assume that…
In the optimization of dynamic systems, the variables typically have constraints. Such problems can be modeled as a Constrained Markov Decision Process (CMDP). This paper considers the peak Constrained Markov Decision Process (PCMDP), where…
Low-fidelity analytical models of turbine wakes have traditionally been used for wind farm planning, performance evaluation, and demonstrating the utility of advanced control algorithms in increasing the annual energy production. In…
We consider power allocation for an access-controlled transmitter with energy harvesting capability based on causal observations of the channel fading state. We assume that the system operates in a time-slotted fashion and the channel gain…
The co-optimization of behind-the-meter distributed energy resources is considered for prosumers under the net energy metering tariff. The distributed energy resources considered include renewable generations, flexible demands, and battery…
This paper deals with unconstrained discounted continuous-time Markov decision processes in Borel state and action spaces. Under some conditions imposed on the primitives, allowing unbounded transition rates and unbounded (from both above…
In this paper, we consider the finite-state approximation of a discrete-time constrained Markov decision process (MDP) under the discounted and average cost criteria. Using the linear programming formulation of the constrained discounted…
Motivated by wide-ranging applications such as video delivery over networks using Multiple Description Codes, congestion control, and inventory management, we study the state-tracking of a Markovian random process with a known transition…
This paper studies the dynamic programming principle using the measurable selection method for stochastic control of continuous processes. The novelty of this work is to incorporate intermediate expectation constraints on the canonical…
This paper proposes to use stochastic conic programming to address the challenge of large-scale wind power integration to the power system. Multiple wind farms are connected through the voltage source converter (VSC) based multi-terminal DC…
We consider sensor scheduling as the optimal observability problem for partially observable Markov decision processes (POMDP). This model fits to the cases where a Markov process is observed by a single sensor which needs to be dynamically…
With the simultaneous rise of energy costs and demand for cloud computing, efficient control of data centers becomes crucial. In the data center control problem, one needs to plan at every time step how many servers to switch on or off in…
We revisit closed-loop performance guarantees for Model Predictive Control in the deterministic and stochastic cases, which extend to novel performance results applicable to receding horizon control of Partially Observable Markov Decision…
Modeling wind speed is one of the key element when dealing with the production of energy through wind turbines. A good model can be used for forecasting, site evaluation, turbines design and many other purposes. In this work we are…
In this paper we study a periodic-review single-commodity setup-cost inventory model with backorders and holding/backlog costs satisfying quasiconvexity assumptions. We show that the Markov decision process for this inventory model…
We introduce StoDynProg, a small library created to solve Optimal Control problems arising in the management of Renewable Power Sources, in particular when coupled with an Energy Storage System. The library implements generic Stochastic…
In this manuscript we continue the thread of [M. Chertkov, F. Pan, M. Stepanov, Predicting Failures in Power Grids: The Case of Static Overloads, IEEE Smart Grid 2011] and suggest a new algorithm discovering most probable extreme stochastic…
We consider the problem of controlling a fully specified Markov decision process (MDP), also known as the planning problem, when the state space is very large and calculating the optimal policy is intractable. Instead, we pursue the more…
We present an alternative view for the study of optimal control of partially observed Markov Decision Processes (POMDPs). We first revisit the traditional (and by now standard) separated-design method of reducing the problem to fully…
This paper proposes a novel energy storage price arbitrage algorithm combining supervised learning with dynamic programming. The proposed approach uses a neural network to directly predicts the opportunity cost at different energy storage…