Related papers: Tejas Simulator : Validation against Hardware
A MICRO 2024 best paper runner-up publication (the Mess paper) with all three artifact badges awarded (including ``Reproducible'') proposes a new benchmark to evaluate real and simulated memory system performance. The publication contends…
Watermarking is a promising defense against the misuse of large language models (LLMs), yet it remains vulnerable to scrubbing and spoofing attacks. This vulnerability stems from an inherent trade-off governed by watermark window size:…
Evaluation of novel treatment effect estimators frequently relies on simulation studies lacking formal statistical comparisons and using arbitrary numbers of replications ($J$). This hinders reproducibility and efficiency. We propose the…
CONTEXT: There is growing interest in establishing software engineering as an evidence-based discipline. To that end, replication is often used to gain confidence in empirical findings, as opposed to reproduction where the goal is showing…
SystemVerilog Assertions (SVAs) play a critical role in detecting and debugging functional bugs in digital chip design. However, generating SVAs has traditionally been a manual, labor-intensive, and error-prone process. Recent advances in…
Modelling & Simulation (M&S) is broadly used in real scenarios where making physical modifications could be highly expensive. With the so-called Simulation Software-as-a-Service (SimSaaS), researchers could take advantage of the huge amount…
Modern microprocessors depend on speculative execution, creating vulnerabilities that enable transient execution attacks. Prior defenses target speculative data leakage but overlook false dependencies from partial address aliasing, where…
Submarines are vital for maritime defense, requiring optimized hydrodynamic performance to minimize resistance. Advancements in Computational Fluid Dynamics (CFD) enable accurate predictions of submarine hydrodynamics for optimal design.…
CERBERUS is a synthetic benchmark designed to help train and evaluate AI models for detecting cracks and other defects in infrastructure. It includes a crack image generator and realistic 3D inspection scenarios built in Unity. The…
Coding agents increasingly act as codebase-scale collaborators that can assist with codebase conversion, but this progress has exposed a critical weakness: agents often over-trust their own local validation routines and declare success on…
Most popular, modern network simulators, such as ns, are targeted towards simulating low-level protocol details. These existing simulators are not intended for simulating large distributed applications with many hosts and many concurrent…
Software debloating tools seek to improve program security and performance by removing unnecessary code, called bloat. While many techniques have been proposed, several barriers to their adoption have emerged. Namely, debloating tools are…
Benchmarking; by which I mean any computer system that is driven by a controlled workload, is the ultimate in performance testing and simulation. Aside from being a form of institutionalized cheating, it also offer countless opportunities…
Computer models, also known as simulators, can be computationally expensive to run, and for this reason statistical surrogates, known as emulators, are often used. Any statistical model, including an emulator, should be validated before…
Memory simulators are used to estimate application performance on advanced memory systems, yet they may exhibit significant discrepancies compared to real hardware. This paper investigates two key questions: (1) what causes these…
The Shear TEsting Programme (STEP) is a collaborative project to improve the accuracy and reliability of weak lensing measurement, in preparation for the next generation of wide-field surveys. We review sixteen current and emerging shear…
Template engines are indispensable components in modern software ecosystems, enabling the generation of structured documents and scripts across domains such as web development, Infrastructure as Code, and data engineering. However, the…
Software Bill of Materials (SBOM) provides new opportunities for automated vulnerability identification in software products. While the industry is adopting SBOM-based Vulnerability Scanning (SVS) to identify vulnerabilities, we…
User simulators are increasingly central to interactive information retrieval, yet the community lacks standardized evaluation tools. Simulators serve two objectives, behavioral realism (matching real user behavior) and tester reliability…
Automatic LLM benchmarks, such as AlpacaEval 2.0, Arena-Hard-Auto, and MT-Bench, have become popular for evaluating language models due to their cost-effectiveness and scalability compared to human evaluation. Achieving high win rates on…