Related papers: WURBench: Toward Benchmarking Wake-up Radio-based …
In recent years, significant progress has been made in solving challenging problems across various domains using deep reinforcement learning (RL). Reproducing existing work and accurately judging the improvements offered by novel methods is…
We present in this paper a new benchmark for evaluating the performances of data warehouses. Benchmarking is useful either to system users for comparing the performances of different systems, or to system engineers for testing the effect of…
Is pushing numbers on a single benchmark valuable in automatic speech recognition? Research results in acoustic modeling are typically evaluated based on performance on a single dataset. While the research community has coalesced around…
We present Task Bench, a parameterized benchmark designed to explore the performance of parallel and distributed programming systems under a variety of application scenarios. Task Bench lowers the barrier to benchmarking multiple…
In recent years, there has been a strong interest in measuring the available bandwidth of network paths. Several methods and techniques have been proposed and various measurement tools have been developed and evaluated. However, there have…
Benchmarking; by which I mean any computer system that is driven by a controlled workload, is the ultimate in performance testing and simulation. Aside from being a form of institutionalized cheating, it also offer countless opportunities…
Wireless sensor networks (WSNs) have emerged as one of the most promising technologies for the current era. Researchers have studied them for several years ago, but more work still needed to be made since open opportunities to integrate new…
WLAN devices have become a fundamental component of nowadays network deployments. However, even though traditional networking applications run mostly unchanged over wireless links, the actual interaction between these applications and the…
In this report we propose a MultiObjective (MO) performance evaluation framework for wireless ad hoc networks where criteria such as capacity, robustness, energy and delay are optimized concurrently. Within such a framework, we can…
In Machine Learning, a benchmark refers to an ensemble of datasets associated with one or multiple metrics together with a way to aggregate different systems performances. They are instrumental in (i) assessing the progress of new methods…
Edge computing is the next Internet frontier that will leverage computing resources located near users, sensors, and data stores to provide more responsive services. Therefore, it is envisioned that a large-scale, geographically dispersed,…
In this work we study the performance of a heterogeneous wireless sensor network which consists of 4 different hardware platforms (TelosB, SunSPOT, Arduino, iSense). All hardware platforms use 802.15.4 compliant radios. Due to partial…
Ultra-wideband (UWB) positioning has emerged as a low-cost and dependable localization solution for multiple use cases, from mobile robots to asset tracking within the Industrial IoT. The technology is mature and the scientific literature…
In this paper, we design a simple, low-cost, and low-power wake-up receiver which can be used for an IEEE 802.11-compliant device to remotely wake up the other devices by utilizing its own wireless LAN (WLAN) signals. The employed wake-up…
Recent Weak Supervision (WS) approaches have had widespread success in easing the bottleneck of labeling training data for machine learning by synthesizing labels from multiple potentially noisy supervision sources. However, proper…
Future wireless systems are expected to support mission-critical services demanding higher and higher reliability. In this letter, we dimension the radio resources needed to achieve a given failure probability target for ultra-reliable…
Randomized benchmarking is a powerful technique to efficiently estimate the performance and reliability of quantum gates, circuits and devices. Here we propose to perform randomized benchmarking in a coherent way, where superpositions of…
Whether it is crosstalk, harmonics, or in-band operation of wireless technologies, interference between a reference system and a host of offenders is virtually unavoidable. In past contributions, a benchmark has been established and…
In empirical software engineering, benchmarks can be used for comparing different methods, techniques and tools. However, the recent ACM SIGSOFT Empirical Standards for Software Engineering Research do not include an explicit checklist for…
A significant amount of work is invested in human-machine teaming (HMT) across multiple fields. Accurately and effectively measuring system performance of an HMT is crucial for moving the design of these systems forward. Metrics are the…