Related papers: WURBench: Toward Benchmarking Wake-up Radio-based …
A critical yet frequently overlooked challenge in the field of deepfake detection is the lack of a standardized, unified, comprehensive benchmark. This issue leads to unfair performance comparisons and potentially misleading results.…
The increasing popularity of ultra-wideband (UWB) technology for location-based services, such as access control and real-time indoor track&tracing, as well as UWB support in new consumer devices such as smartphones, has resulted in the…
Transferability estimation metrics are used to find a high-performing pre-trained model for a given target task without fine-tuning models and without access to the source dataset. Despite the growing interest in developing such metrics,…
Wireless Technology Recognition (WTR) and localization are essential in modern communication systems, enabling efficient spectrum management, seamless coexistence of diverse technologies, and accurate positioning in dynamic environments. In…
The performance of a wireless sensor network (WSN) depends fundamentally on how its various parameters are configured under different link quality conditions. Surprisingly, even though WSNs have been extensively researched, there still…
Randomized benchmarking is a promising tool for characterizing the noise in experimental implementations of quantum systems. In this paper, we prove that the estimates produced by randomized benchmarking (both standard and interleaved) for…
Agreeing suitability for purpose and procurement decisions depend on assessment of real or simulated performances of sonar systems against user requirements for particular scenarios. There may be multiple pertinent aspects of performance…
We present Benchpress, a benchmarking suite for evaluating the performance and range of functionality of multiple quantum computing software development kits. This suite consists of a collection of over $1000$ tests measuring key…
Programmable wireless environments enable the software-defined propagation of waves within them, yielding exceptional performance potential. Several building-block technologies have been implemented and evaluated at the physical layer. The…
The need for performance measurement tools appeared soon after the emergence of the first Object-Oriented Database Management Systems (OODBMSs), and proved important for both designers and users (Atkinson \& Maier, 1990). Performance…
Recent advances in probabilistic modelling have led to a large number of simulation-based inference algorithms which do not require numerical evaluation of likelihoods. However, a public benchmark with appropriate performance metrics for…
For scientific software, especially those used for large-scale simulations, achieving good performance and efficiently using the available hardware resources is essential. It is important to regularly perform benchmarks to ensure the…
Over the past decade, U-Net has been the dominant architecture in medical image segmentation, leading to the development of thousands of U-shaped variants. Despite its widespread adoption, there is still no comprehensive benchmark to…
The radio spectrum is characterized by a noticeable variability, which impairs performance and determinism of every wireless communication technology. To counteract this aspect, mechanisms like Minstrel are customarily employed in real…
Current benchmarks are inadequate for evaluating progress in reinforcement learning (RL) for large language models (LLMs).Despite recent benchmark gains reported for RL, we find that training on these benchmarks' training sets achieves…
Quantitative Artificial Intelligence (AI) Benchmarks have emerged as fundamental tools for evaluating the performance, capability, and safety of AI models and systems. Currently, they shape the direction of AI development and are playing an…
Large-scale Internet of Things (IoT) applications, such as asset tracking and remote sensing, demand multi-year battery lifetimes to minimize maintenance and operational costs. Traditional wireless protocols often employ duty cycling,…
AI models are increasingly prevalent in high-stakes environments, necessitating thorough assessment of their capabilities and risks. Benchmarks are popular for measuring these attributes and for comparing model performance, tracking…
Wireless Mesh Network (WMN) is surely one of the prominent networks in the modern era which is widely used in numerous evolving applications, viz. broadband home networking (BHN), community and neighbourhood networks (CNN), coordinated…
In the burgeoning age of generative AI, watermarks act as identifiers of provenance and artificial content. We present WAVES (Watermark Analysis Via Enhanced Stress-testing), a benchmark for assessing image watermark robustness, overcoming…