相关论文: A Learning-based Optimal Market Bidding Strategy f…

In this paper, we present the use of Model Predictive Control (MPC) based on Reinforcement Learning (RL) to find the optimal policy for a multi-agent battery storage system. A time-varying prediction of the power price and production-demand…

This paper introduces and rationalizes a new model for bidding and clearing energy storage resources in wholesale energy markets. Charge and discharge bids in this model depend on the storage state-of-charge (SoC). In this setting, storage…

In a day-ahead market, energy buyers and sellers submit their bids for a particular future time, including the amount of energy they wish to buy or sell and the price they are prepared to pay or receive. However, the dynamic for forming the…

Soft Actor-Critic (SAC) is an off-policy actor-critic reinforcement learning algorithm, essentially based on entropy regularization. SAC trains a policy by maximizing the trade-off between expected return and entropy (randomness in the…

We consider the problem of co-optimized energy-reserve market clearing with state-of-charge (SoC) dependent bids from battery storage participants. While SoC-dependent bids capture storage's degradation and opportunity costs, such bids…

The paper explores the application of a continuous action space soft actor-critic (SAC) reinforcement learning model to the area of automated market-making. The reinforcement learning agent receives a simulated flow of client trades, thus…

Actor-critic (AC) algorithms are known for their efficacy and high performance in solving reinforcement learning problems, but they also suffer from low sampling efficiency. An AC based policy optimization process is iterative and needs to…

We focus on a simulation-based optimization problem of choosing the best design from the feasible space. Although the simulation model can be queried with finite samples, its internal processing rule cannot be utilized in the optimization…

We consider the problem of demand-side energy management, where each household is equipped with a smart meter that is able to schedule home appliances online. The goal is to minimize the overall cost under a real-time pricing scheme. While…

Energy market rules should incentivize market participants to behave in a market and grid conform way. However, they can also provide incentives for undesired and unexpected strategies if the market design is flawed. Multi-agent…

The application of deep reinforcement learning algorithms to economic battery dispatch problems has significantly increased recently. However, optimizing battery dispatch over long horizons can be challenging due to delayed rewards. In our…

The Pay-as-Clear (PaC) mechanism currently used in the European electricity market can generate significant submarginal profits for renewable sources when the clearing price is determined by the marginal offers of gas-fired generation units…

The cost of the power distribution infrastructures is driven by the peak power encountered in the system. Therefore, the distribution network operators consider billing consumers behind a common transformer in the function of their peak…

With the advances in the Internet of Things technology, electric vehicles (EVs) have become easier to schedule in daily life, which is reshaping the electric load curve. It is important to design efficient charging algorithms to mitigate…

Exploration in multi-agent reinforcement learning is a challenging problem, especially in environments with sparse rewards. We propose a general method for efficient exploration by sharing experience amongst agents. Our proposed algorithm,…

Model-predictive-control (MPC) offers an optimal control technique to establish and ensure that the total operation cost of multi-energy systems remains at a minimum while fulfilling all system constraints. However, this method presumes an…

Electricity markets are experiencing a rapid increase in energy storage unit participation. Unlike conventional generation resources, quantifying the competitive operation and identifying if a storage unit is exercising market power is…

We consider an improper reinforcement learning setting where a learner is given $M$ base controllers for an unknown Markov decision process, and wishes to combine them optimally to produce a potentially new controller that can outperform…

Energy storage is expected to play an increasingly important role in mitigating variations that come along with the growing penetration of renewable energy. In this paper, we study the optimal bidding of an energy storage unit in a…

The design of energy markets is a subject of ongoing debate, particularly concerning the choice between the widely adopted Pay-as-Clear (PC) pricing mechanism and the alternative Pay-as-Bid (PB). These mechanisms determine how energy…