Related papers: WURBench: Toward Benchmarking Wake-up Radio-based …
The use of Wake-Up Radio (WUR) in Internet of Things (IoT) networks can significantly improve their energy efficiency: battery-powered sensors can remain in a low-power (sleep) mode while listening for wake-up messages using their WUR and…
Quantum Computing (QC) is undergoing a high rate of development, investment and research devoted to its improvement.However, there is little consensus in the industry and wider literature as to what improvement might consist of beyond…
Benchmarking has long served as a foundational practice in machine learning and, increasingly, in modern AI systems such as large language models, where shared tasks, metrics, and leaderboards offer a common basis for measuring progress and…
Recent advancements in ultra-low-power machine learning (TinyML) hardware promises to unlock an entirely new class of smart applications. However, continued progress is limited by the lack of a widely accepted benchmark for these systems.…
Quantum computers have the potential to provide an advantage over classical computers in a number of areas. Numerous metrics to benchmark the performance of quantum computers, ranging from their individual hardware components to entire…
The systems based on intelligent sensors are currently expanding, due to theirs functions and theirs performances of intelligence: transmitting and receiving data in real-time, computation and processing algorithms, metrology remote,…
Neuromorphic computing shows promise for advancing computing efficiency and capabilities of AI applications using brain-inspired principles. However, the neuromorphic research field currently lacks standardized benchmarks, making it…
Rigorous and reproducible evaluation is critical for assessing the state of the art and for guiding scientific advances in Artificial Intelligence. Evaluation is challenging in practice due to several reasons, including benchmark…
We present a transmission scheme aimed at integrating wake-up radios (WuR) into LoRa sensor networks featuring UAV-mounted gateways. The sensors are informed of the UAV's arrival with the help of WuR, followed by sensor-to-UAV data transfer…
Enhancing the energy efficiency of devices stands as one of the key requirements in the fifth-generation (5G) cellular network and its evolutions toward the next generation wireless technology. Specifically, for battery-limited…
The metrology field has been progressed with the appearance of the wireless intelligent sensor systems providing more capabilities such as signal processing, remote multi-sensing fusion etc. This kind of devices is rapidly making their way…
Benchmarking is how the performance of a computing system is determined. Surprisingly, even for classical computers this is not a straightforward process. One must choose the appropriate benchmark and metrics to extract meaningful results.…
Benchmarking and establishing proper statistical validation metrics for reinforcement learning (RL) remain ongoing challenges, where no consensus has been established yet. The emergence of quantum computing and its potential applications in…
As quantum computing systems continue to mature, there is an increasing need for benchmarking methodologies that capture performance in terms of meaningful, application-level metrics. In this work, we present a scalable framework for…
Transferability scores aim to quantify how well a model trained on one domain generalizes to a target domain. Despite numerous methods proposed for measuring transferability, their reliability and practical usefulness remain inconclusive,…
We introduce a comprehensive framework for modeling single cell transcriptomic responses to perturbations, aimed at standardizing benchmarking in this rapidly evolving field. Our approach includes a modular and user-friendly model…
Wakeup is the primary function in voice interaction which is the mainstream scheme in man-machine interaction (HMI) applications for smart home. All devices will response if the same wake-up word is used for all devices. This will bring…
Wireless communications and sensing (WCS) establish the backbone of modern information exchange and environment perception. Typical applications range from mobile networks and the Internet of Things to radar and sensor grids. Despite…
In this work, we develop a measurement platform to capture mobile network performance metrics including coverage and quality of service in regions where conventional coverage testing approaches are frequently time-intensive,…
AI models are increasingly deployed in live clinical environments where they must perform reliably across complex, high-stakes workflows that standard training and validation datasets were never designed to capture. Evaluating these systems…