Search papers, labs, and topics across Lattice.
This paper introduces a label-free governance monitoring extension to the Governance Drift Toolkit for risk decision systems, addressing the challenge of silent model performance degradation during the blind period before ground truth labels are available. The approach uses composite multi-proxy monitoring across score distribution, feature drift, prediction entropy, and confidence distribution, with governance-calibrated thresholds to generate governance alerts. Empirical evaluation on a credit scoring dataset demonstrates the ability to distinguish covariate degradation from natural drift and reveals the inherent limitations of label-free monitoring in detecting pure concept drift.
You can now detect governance evidence degradation in risk decision systems *without* labels, but be warned: pure concept drift remains undetectable.
Risk decision systems in fraud detection and credit scoring operate under structural label absence: ground truth arrives weeks to months after decisions are made. During this blind period, model performance may degrade silently, eroding the governance evidence that justifies automated decisions. Existing drift detection methods either require labels (supervised detectors) or detect statistical change without distinguishing harmful degradation from benign distributional evolution (unsupervised detectors). No existing framework integrates drift detection with governance evidence assessment and operational response. This paper presents a label-free governance monitoring extension to the Governance Drift Toolkit that produces governance alerts rather than statistical alarms. The monitoring architecture applies composite multi-proxy monitoring across four proxy monitors (score distribution, feature drift, prediction entropy, confidence distribution), with governance-calibrated thresholds. Empirical evaluation on the Lending Club credit scoring dataset (1.37M loans, 11 years) demonstrates three findings. First, raw proxy metrics (Feature PSI delta up to 1.84, Score PSI delta up to 0.92) distinguish injected covariate degradation from natural temporal drift in an offline evaluation setting. Second, pure concept drift in P(Y|X) produces exactly zero delta across all proxy metrics in all windows, confirming the irreducible blind spot of label-free monitoring as a structural verification. Third, the composite score provides monotonic severity progression as more monitors trigger (0.583 to 0.833 to 1.000), enabling graduated governance response. Cross-domain comparison with IEEE-CIS fraud detection results shows the detectable/undetectable boundary is consistent across both domains. The toolkit and evaluation code are available as open-source artifacts.
