Importing relationships into a running graph database using multiple threads running concurrently is a difficult task, as multiple threads cannot write information to the same node at the same time. Here we present an algorithm in which relationships are sorted into bins, then imported such that no two threads ever access the same node concurrently. When this algorithm was implemented as a procedure to run on the Neo4j graph database, it reduced the time to import relationships by up to 69% when 32 threads were used.
@article{arxiv.2005.04093,
title = {Importing Relationships into a Running Graph Database Using Parallel Processing},
author = {Joshua Porter and Aleks Ontman},
journal= {arXiv preprint arXiv:2005.04093},
year = {2020}
}
Comments
5 pages, code provided on GitHub https://github.com/Lnofeisone/graph-iterateRelationship