Search papers, labs, and topics across Lattice.
This study investigates the reliability of machine-extracted legal logic by analyzing the divergence in outputs from two independent parsers on Missouri's statutes, revealing a significant false-negative rate of 0.43. A passive survival certificate is introduced to certify the robustness of machine-extracted implications based on the Duquenne-Guigues implication basis, using Monte Carlo simulations to measure inter-extractor disagreement. Despite achieving a high certification rate across a large dataset, the findings highlight that a substantial portion of the output remains below acceptable informativeness thresholds, indicating challenges in practical deployment.
Machine-extracted legal implications can be certified for reliability, but nearly 93% of outputs may still lack sufficient informativeness under real-world error conditions.
Statutes are increasingly parsed by machines before people read them, and the parsers disagree: on Missouri's statutes, two independently written extractors diverge on numeric-threshold presence at a false-negative rate of 0.43. We ask what formal logic survives such noise. We build a passive survival certificate for the Duquenne-Guigues implication basis of machine-extracted statutory contexts: per-attribute inter-extractor disagreement is measured, replayed against the basis in 1,000 Monte Carlo trials, and an implication is certified only when a one-sided Wilson 95% lower bound on survival reaches 0.95; every certified implication carries premise spans and a minimal counterexample. On 29,365 Missouri sections and 502 Indian central-Act sections, the preregistered held-out gate passes (10 statute families across 7 Titles exact; 16 across 11 with 5% tolerance), yet under one globally deployed error model 93.2% of held-out chapters fall below the informativeness floor, and a 2x2 factorial assigns that to calibration-rate transfer, not selection. The certificate is usable but fragile: deploy it per-chapter-calibrated or error-tolerant. Code, data products, and the audit trail, including one retracted claim, are released.
