With the escalating complexity and volume of genomic data, the capacity of biology institutions' HPC faces limitations. While the Cloud presents a viable solution for short-term elasticity, its intricacies pose challenges for bioinformatics users. Alternatively, serverless computing allows for workload scalability with minimal developer burden. However, porting a scientific application to serverless is not a straightforward process. In this article, we present a Variant Calling genomics pipeline migrated from single-node HPC to a serverless architecture. We describe the inherent challenges of this approach and the engineering efforts required to achieve scalability. We contribute by open-sourcing the pipeline for future systems research and as a scalable user-friendly tool for the bioinformatics community.
@article{arxiv.2312.07090,
title = {Scaling a Variant Calling Genomics Pipeline with FaaS},
author = {Aitor Arjona and Arnau Gabriel-Atienza and Sara Lanuza-Orna and Xavier Roca-Canals and Ayman Bourramouss and Tyler K. Chafin and Lucio Marcello and Paolo Ribeca and Pedro García-López},
journal= {arXiv preprint arXiv:2312.07090},
year = {2023}
}
Comments
6 pages, published at 9th International Workshop on Serverless Computing (WoSC '23), December 11-15, 2023, Bologna, Italy