English

DataJoint 2.0: A Computational Substrate for Agentic Scientific Workflows

Databases 2026-02-19 v1 Artificial Intelligence

Abstract

Operational rigor determines whether human-agent collaboration succeeds or fails. Scientific data pipelines need the equivalent of DevOps -- SciOps -- yet common approaches fragment provenance across disconnected systems without transactional guarantees. DataJoint 2.0 addresses this gap through the relational workflow model: tables represent workflow steps, rows represent artifacts, foreign keys prescribe execution order. The schema specifies not only what data exists but how it is derived -- a single formal system where data structure, computational dependencies, and integrity constraints are all queryable, enforceable, and machine-readable. Four technical innovations extend this foundation: object-augmented schemas integrating relational metadata with scalable object storage, semantic matching using attribute lineage to prevent erroneous joins, an extensible type system for domain-specific formats, and distributed job coordination designed for composability with external orchestration. By unifying data structure, data, and computational transformations, DataJoint creates a substrate for SciOps where agents can participate in scientific workflows without risking data corruption.

Keywords

Cite

@article{arxiv.2602.16585,
  title  = {DataJoint 2.0: A Computational Substrate for Agentic Scientific Workflows},
  author = {Dimitri Yatsenko and Thinh T. Nguyen},
  journal= {arXiv preprint arXiv:2602.16585},
  year   = {2026}
}

Comments

20 pages, 2 figures, 1 table

R2 v1 2026-07-01T10:41:34.424Z