English

SerPyTor: A distributed context-aware computational graph execution framework for durable execution

Distributed, Parallel, and Cluster Computing 2023-04-18 v1

Abstract

Distributed computation is always a tricky topic to deal with, especially in context of various requirements in various scenarios. A popular solution is to use Apache Spark with a setup of multiple systems forming a cluster. However, the prerequisite setup for a Spark cluster often induces an additional overhead, often limiting usage in constrained scenarios, especially in scenarios requiring context propagation. In this paper, we explore a relatively lightweight computational graph execution framework requiring little setup and fast speeds, coupled with context awareness.

Keywords

Cite

@article{arxiv.2304.07555,
  title  = {SerPyTor: A distributed context-aware computational graph execution framework for durable execution},
  author = {Anuran Roy and Sridhar Raj S},
  journal= {arXiv preprint arXiv:2304.07555},
  year   = {2023}
}

Comments

5 pages, 2 figures