Search papers, labs, and topics across Lattice.
DataJoint 2.0 introduces a relational workflow model for SciOps, representing workflow steps as tables, artifacts as rows, and execution order via foreign keys to ensure provenance and transactional guarantees. This system unifies data structure, data, and computational transformations within a single formal system, enabling queryable, enforceable, and machine-readable scientific workflows. The key result is a computational substrate that facilitates human-agent collaboration in scientific workflows while preventing data corruption through object-augmented schemas, semantic matching, an extensible type system, and distributed job coordination.
DataJoint 2.0 lets AI agents participate in scientific workflows without risking data corruption, by unifying data structure, data, and computational transformations.
Operational rigor determines whether human-agent collaboration succeeds or fails. Scientific data pipelines need the equivalent of DevOps -- SciOps -- yet common approaches fragment provenance across disconnected systems without transactional guarantees. DataJoint 2.0 addresses this gap through the relational workflow model: tables represent workflow steps, rows represent artifacts, foreign keys prescribe execution order. The schema specifies not only what data exists but how it is derived -- a single formal system where data structure, computational dependencies, and integrity constraints are all queryable, enforceable, and machine-readable. Four technical innovations extend this foundation: object-augmented schemas integrating relational metadata with scalable object storage, semantic matching using attribute lineage to prevent erroneous joins, an extensible type system for domain-specific formats, and distributed job coordination designed for composability with external orchestration. By unifying data structure, data, and computational transformations, DataJoint creates a substrate for SciOps where agents can participate in scientific workflows without risking data corruption.
