English

CARGO: Effective format-free compressed storage of genomic information

Genomics 2021-11-01 v1 Computational Engineering, Finance, and Science Data Structures and Algorithms

Abstract

The recent super-exponential growth in the amount of sequencing data generated worldwide has put techniques for compressed storage into the focus. Most available solutions, however, are strictly tied to specific bioinformatics formats, sometimes inheriting from them suboptimal design choices; this hinders flexible and effective data sharing. Here we present CARGO (Compressed ARchiving for GenOmics), a high-level framework to automatically generate software systems optimized for the compressed storage of arbitrary types of large genomic data collections. Straightforward applications of our approach to FASTQ and SAM archives require a few lines of code, produce solutions that match and sometimes outperform specialized format-tailored compressors, and scale well to multi-TB datasets.

Keywords

Cite

@article{arxiv.1506.05185,
  title  = {CARGO: Effective format-free compressed storage of genomic information},
  author = {Łukasz Roguski and Paolo Ribeca},
  journal= {arXiv preprint arXiv:1506.05185},
  year   = {2021}
}

Comments

13 (Main) + 31 (Supplementary) + 88 (Manual) pages