Gene regulatory networks are powerful tools for modeling interactions among genes to regulate their expression for homeostasis and differentiation. Single-cell sequencing offers a unique opportunity to build these networks with high-resolution data. There are many proposed computational methods to build these networks using single-cell data and different approaches are followed to benchmark these methods. In this review, we lay the basic terminology in the field and define the success metrics. Next, we present an overview of approaches for benchmarking computational gene regulatory network approaches for building gene regulatory networks and point out gaps and future directions in this regard.