Search papers, labs, and topics across Lattice.
This paper conducts a multi-surface consistency audit of citation metadata across 117 open-source research software projects to assess the agreement among various self-descriptions. The findings reveal that 83.9% of projects with multiple metadata surfaces exhibit at least one core-field conflict, primarily due to discrepancies in how software and its associated papers are described. By highlighting these inconsistencies, the study underscores the potential for fragmented credit and provenance in research software, which can impact attribution and recognition in the scientific community.
Over 80% of research software projects have conflicting metadata across their self-descriptions, risking fragmented credit and provenance in the scientific community.
Research software projects describe themselves in many places at once: citation files in the repository, archive deposits, DOI registry records, package registries, and README text. We treat the software as the underlying object and these machine-readable self-descriptions as its surfaces: the points where people and automated systems read what the project declares about the software. Citation guidance, indexing services, and automated agents may read a different subset of these surfaces, so disagreement between them can silently fragment credit and provenance. This paper asks a simple question that has not been measured directly: when a project's own metadata surfaces are compared with each other, how often do they agree? We audited 117 open-source research software projects, comprising an 87-project high-performance computing and quantum computing corpus and a 30-project registered baseline drawn from the JOSS and pyOpenSci accepted-package lists, across up to seven machine-readable surfaces per project. Using a four-level verdict rubric across six metadata fields, with 98.5\% hand-verified verdict precision on a 338-row stratified sample, we found that 52 of the 62 projects exposing at least two comparable surfaces (83.9\%) contain at least one core-field conflict, a result that is insensitive to the fuzzy-matching threshold. Half of hand-adjudicated cross-surface conflicts trace to a single mechanism: surfaces describing the software's paper rather than the software itself. Among projects whose CITATION.cff includes a preferred citation, 28 of 32 route citations to a record that disagrees with the software's own metadata. The author lists and titles disagree the most, and the registry surfaces are the least aligned. We release the audit pipeline as an importable library, the corpus, the registered sampling protocol, all raw snapshots, and the complete verification log.
