When a software transformation or software security task needs to analyze a given program binary, the first step is often disassembly. Since many modern disassemblers have become highly accurate on many binaries, we believe reliable disassembler benchmarking requires standardizing the set of binaries used and the disassembly ground truth about these binaries. This paper presents (i) a first version of our work-in-progress disassembly benchmark suite, which comprises 879 binaries from diverse projects compiled with multiple compilers and optimization settings, and (ii) a novel disassembly ground truth generator leveraging the notion of "listing files", which has broad support by Clang, GCC, ICC, and MSVC. In additional, it presents our evaluation of four prominent open-source disassemblers using this benchmark suite and a custom evaluation system. Our entire system and all generated data are maintained openly on GitHub to encourage community adoption.
@article{arxiv.2012.09155,
title = {On the Generation of Disassembly Ground Truth and the Evaluation of Disassemblers},
author = {Kaiyuan Li and Maverick Woo and Limin Jia},
journal= {arXiv preprint arXiv:2012.09155},
year = {2020}
}
Comments
Revised and extended version of our publication that first appeared in the 2020 Workshop on Forming an Ecosystem Around Software Transformation (FEAST '20), November 13, 2020