Search papers, labs, and topics across Lattice.
This study investigates the effectiveness of structure-preserving de-identification in maintaining the detectability of protected health information (PHI) by surrogate substitutes across multiple detectors. Utilizing a paired, multi-detector evaluation protocol, the authors demonstrate that the recall of PHI detection remains statistically equivalent before and after surrogate substitution, with only a minor decrease from 76.1% to 74.9%. The findings reveal that the slight loss in detection accuracy is primarily due to issues with malformed surrogates rather than a decline in detector performance, emphasizing the importance of well-formed substitutions in clinical text.
Surrogate substitutions in de-identification maintain PHI detectability across multiple detectors, with a negligible impact on recall.
Structure-preserving de-identification replaces protected health information (PHI) with realistic same-type surrogates --"Anna S."becomes"Maria S.", not [NAME] -- so that clinical text stays fluent and downstream tools keep working. But this only helps if the substitution does not itself corrupt the signal those tools rely on. We ask a narrow, testable question: on the spans a de-identifier actually masks, can downstream PHI detectors still find the surrogate? We introduce a paired, multi-detector evaluation protocol that (i) scores utility only on masked spans, decoupling coverage from utility; (ii) uses equivalence testing (TOST) rather than null-hypothesis significance testing, which is uninformative at our sample size (57k paired spans); and (iii) builds a surrogate-failure typology separating fixable generator defects from intrinsic detector limits. Across 11 detectors, 7 benchmarks, and 7 languages (1,750 documents), recall on masked spans moves from 76.1% to 74.9% -- a change our equivalence test shows is statistically equivalent to zero within a +/-2-point margin (p ~ 3e-9), with detector ranking preserved. The residual loss does not reflect detectors getting worse at PHI: it concentrates in malformed and out-of-distribution surrogates (truncation Chicago ->Illino, salience loss Cedars-Sinai ->Vidant). A redaction floor and an open-source surrogate baseline indicate the effect is a property of well-formed substitution, not of one tool. We release the evaluation subsets, scoring code, and an interactive dashboard at https://custodianai.pages.dev so the protocol can audit any structure-preserving transform.
