Unshuffling fields in data formats
Information Theory
2020-03-03 v2 Discrete Mathematics
math.IT
Abstract
Data format reverse engineering commonly involves identifying conserved format motifs. However, this process typically requires establishing a common ordering for format elements across instances, particularly for formats using type-(length)-value tuples or "chunk" encoding. It is useful to \emph{unshuffle} chunks with common length statistics as a precursor to identifying conserved internal structures. We formalize the unshuffling problem and subsequently derive probabilistic bounds and outline corresponding algorithms for it. We empirically demonstrate unshuffling and highlight connections with the related class of synchronization problems.
Keywords
Cite
@article{arxiv.1910.09964,
title = {Unshuffling fields in data formats},
author = {Steve Huntsman},
journal= {arXiv preprint arXiv:1910.09964},
year = {2020}
}
Comments
6 pages; 7 figures