Related papers: Energy-Arena: A Dynamic Benchmark for Operational …
As the adoption of Generative AI in real-world services grow explosively, energy has emerged as a critical bottleneck resource. However, energy remains a metric that is often overlooked, under-explored, or poorly understood in the context…
An important goal of modern scheduling systems is to efficiently manage power usage. In energy-efficient scheduling, the operating system controls the speed at which a machine is processing jobs with the dual objective of minimizing energy…
Research on Artificial Intelligence (AI)-based Data Assimilation (DA) is expanding rapidly. However, the absence of an objective, comprehensive, and real-world benchmark hinders the fair comparison of diverse methods. Here, we introduce…
Energy forecasting is an essential task in power system operations. Operators usually issue forecasts and leverage them to schedule energy dispatch ahead of time. However, forecast models are typically developed in a way that overlooks the…
The rapid evolution of Large Language Models (LLMs) is predicated on the quality and diversity of post-training datasets. However, a critical dichotomy persists: while models are rigorously benchmarked, the data fueling them remains a black…
A novel framework for hierarchical forecast updating is presented, addressing a critical gap in the forecasting literature. By assuming a temporal hierarchy structure, the innovative approach extends hierarchical forecast reconciliation to…
Benchmark quality is critical for meaningful evaluation and sustained progress in time series forecasting, particularly with the rise of pretrained models. Existing benchmarks often have limited domain coverage or overlook real-world…
Energy forecasting is pivotal in energy systems, by providing fundamentals for operation, with different horizons and resolutions. Though energy forecasting has been widely studied for capturing temporal information, very few works…
Driven by the transition towards a climate-neutral energy system, accurate energy time series forecasting is critical for planning and operation. Yet, it remains largely a dataset-specific task, requiring comprehensive training data,…
Because of increasing amounts of intermittent and distributed generators in power systems, many demand response programs have been developed to schedule flexible energy consumption. However, proper benchmarks for comparing these methods are…
The increasing use of renewable energy sources with variable output, such as solar photovoltaic and wind power generation, calls for Smart Grids that effectively manage flexible loads and energy storage. The ability to forecast consumption…
Time series forecasting has applications across domains and industries, especially in healthcare, but the technical expertise required to analyze data, build models, and interpret results can be a barrier to using these techniques. This…
Energy system models require a large amount of technical and economic data, the quality of which significantly influences the reliability of the results. Some of the variables on the important data source ENTSO-E transparency platform, such…
Energy consumption represents a significant cost in data center operation. A large fraction of the energy, however, is used to power idle servers when the workload is low. Dynamic provisioning techniques aim at saving this portion of the…
To meet widely recognised carbon neutrality targets, over the last decade metropolitan regions around the world have implemented policies to promote the generation and use of sustainable energy. Nevertheless, there is an availability gap in…
Recurring industrial analytics and machine-learning workflows are becoming a major computational burden in modern engineering practice. Large parametric database generation, scheduled model retraining, repeated evaluation pipelines, and…
This article aims at facilitating the widespread application of Energy Management Systems (EMSs), especially on buildings and cities, in order to support the realization of future carbon-neutral energy systems. We claim that economic…
This paper introduces FieldWorkArena, a benchmark for agentic AI targeting real-world field work. With the recent increase in demand for agentic AI, they are built to detect and document safety hazards, procedural violations, and other…
The rise of agentic AI systems, where agents collaborate to perform diverse tasks, poses new challenges with observing, analyzing and optimizing their behavior. Traditional evaluation and benchmarking approaches struggle to handle the…
Evaluation plays a crucial role in the advancement of information retrieval (IR) models. However, current benchmarks, which are based on predefined domains and human-labeled data, face limitations in addressing evaluation needs for emerging…