Related papers: Pathwise Optimization for Merchant Energy Producti…

For a Markov decision process with countably infinite states, the optimal value may not be achievable in the set of stationary policies. In this paper, we study the existence conditions of an optimal stationary policy in a countable-state…

This paper studies the problem of optimally extracting nonrenewable natural resources. Taking into account the fact that the market values of the main natural resources i.e. oil, natural gas, copper,..., etc, fluctuate randomly following…

We propose a novel randomized linear programming algorithm for approximating the optimal policy of the discounted Markov decision problem. By leveraging the value-policy duality and binary-tree data structures, the algorithm adaptively…

In this paper, we consider solving discounted Markov Decision Processes (MDPs) under the constraint that the resulting policy is stabilizing. In practice MDPs are solved based on some form of policy approximation. We will leverage recent…

Many applications -- including power systems, robotics, and economics -- involve a dynamical system interacting with a stochastic and hard-to-model environment. We adopt a reinforcement learning approach to control such systems.…

Demand-side energy management, such as the real-time pricing (RTP) program, offers manufacturers opportunities to reduce energy costs by shifting production to low-price hours. However, this strategy is challenging to implement when machine…

This article presents a constrained policy optimization approach for the optimal control of systems under nonstationary uncertainties. We introduce an assumption that we call Markov embeddability that allows us to cast the stochastic…

A two-stage multi-period mixed-integer linear stochastic programming model is proposed to assist qualified operators in long-term generation and transmission expansion planning of electricity and gas systems to meet policy objectives. The…

In this paper, we propose an economic nonlinear model predictive control (MPC) algorithm for district heating networks (DHNs). The proposed method features prosumers, multiple producers, and storage systems, which are essential components…

We introduce a new method, stepwise method for solving optimal con- trol problems. Our first motivation for new approach emanate from limi- tations on continuous time control functions in PMP. Practically in most of the real world models,…

In this paper, we investigate an energy cost minimization problem for prosumers participating in peer-to-peer energy trading. Due to (i) uncertainties caused by renewable energy generation and consumption, (ii) difficulties in developing an…

The industrial production of microalgae is an important and sustainable process, but its actual competitiveness is closely related to its optimization. The biological nature of the process hinders this task, mainly due to the high…

One of the major limitations of optimization-based strategies for allocating the power flow in hybrid powertrains is that they rely on predictions of future power demand. These predictions are inherently uncertain as they are dependent on…

We investigate the classical active pure exploration problem in Markov Decision Processes, where the agent sequentially selects actions and, from the resulting system trajectory, aims at identifying the best policy as fast as possible. We…

This paper deals with optimal prediction in a regime-switching model driven by a continuous-time Markov chain. We extend existing results for geometric Brownian motion by deriving optimal stopping strategies that depend on the current…

Position-controlled systems driving repetitive tasks are of significant importance in industrial machinery. The electric actuators used in these systems are responsible for a large part of the global energy consumption, indicating that…

Model predictive control (MPC) has emerged as an effective strategy for water distribution systems (WDSs) management. However, it is hampered by the computational burden for large-scale WDSs due to the combinatorial growth of possible…

We provide an overview on how to use the measurable selection techniques to derive the dynamic programming principle for a general stochastic optimal control/stopping problem. By considering its martingale problem formulation on the…

Edge Computing enables low-latency processing for real-time applications but introduces challenges in power management due to the distributed nature of edge devices and their limited energy resources. This paper proposes a stochastic…

Recently there has been a lot of progress in the development of economic nonlinear model predictive control (NMPC) schemes for multistage optimal power flow (OPF) problems. However, the additional inclusion of discrete decision variables to…