Search papers, labs, and topics across Lattice.
This paper introduces Workflow Cards, a structured documentation format that encapsulates the critical details of workflow executions in machine learning, addressing a significant gap in existing Model and Data Cards. By defining a template based on provenance questions, the authors highlight the importance of documenting execution-level data, which is essential for understanding biases, performance variations, and reproducibility issues. The evaluation reveals that Workflow Cards significantly enhance the quality of information retrieval by nearly doubling the answer quality when compared to traditional schema-based querying methods.
Workflow Cards nearly double the quality of insights from workflow executions, bridging a critical documentation gap in machine learning practices.
Model Cards and Data Cards have demonstrated the value of structured, human-readable documentation for machine learning artifacts, capturing their context, parameters, limitations, and intended use. However, these practices remain focused on static artifacts (the datasets and trained models themselves) while overlooking the workflow executions that produce, transform, and evaluate them. Such executions hold critical details about data preparation, parameter choice, runtime behavior, resource use, and intermediate transformations, precisely where bias, performance variation, and reproducibility gaps tend to originate. To close this gap, we introduce Workflow Cards: structured summaries that condense the machine-readable provenance data of a workflow execution into a form both humans and large language models (LLMs) can read and analyze. This paper has two main parts. First, it defines a Workflow Card template informed by a representative set of provenance questions that surface from the execution-level data missing from Model and Data Cards. Second, it evaluates how effectively LLMs use Workflow Cards to understand workflow executions compared with querying provenance databases through a schema-based interface. Results show that Workflow Cards provide execution-level information absent from existing card types, such as Model Cards and Data Cards, thereby filling an important documentation gap; and that Workflow Cards nearly double answer quality compared with schema-based querying, consistently across LLM-as-a-Judge and human assessments.
