This paper proposes a composable "Just in Time Architecture" for Data Science (DS) Pipelines named JITA-4DS and associated resource management techniques for configuring disaggregated data centers (DCs). DCs under our approach are composable based on vertical integration of the application, middleware/operating system, and hardware layers customized dynamically to meet application Service Level Objectives (SLO - application-aware management). Thereby, pipelines utilize a set of flexible building blocks that can be dynamically and automatically assembled and re-assembled to meet the dynamic changes in the workload's SLOs. To assess disaggregated DC's, we study how to model and validate their performance in large-scale settings.
@article{arxiv.2103.07978,
title = {Putting Data Science Pipelines on the Edge},
author = {Ali Akoglu and Genoveva Vargas-Solar},
journal= {arXiv preprint arXiv:2103.07978},
year = {2021}
}