Search papers, labs, and topics across Lattice.
This study audits the performance of automatic verifiers in reinforcement learning with verifiable rewards (RLVR) by applying metamorphic testing to identify specific answer forms that lead to rejection. The analysis reveals significant variability in self-validation rates across different verifiers, with a notable concentration of errors related to whitespace and punctuation. Key findings indicate that while aggregate error rates may appear similar, the underlying causes of rejection differ dramatically between verifiers, highlighting the need for more nuanced evaluation methods in RLVR systems.
Self-validation rates for RLVR verifiers can vary by over 41% depending on the configuration, with punctuation errors accounting for 93% of failures.
Reinforcement learning with verifiable rewards (RLVR) and standard benchmark evaluation both rely on an automatic verifier that turns a free text answer into a binary reward. Prior work reports that one evaluation harness accepts only about 94% of its own ground truth answers, blaming LaTeX parsing. That is an aggregate: it does not say which answer forms consume the error budget. We supply the decomposition. We apply metamorphic testing to the verifier rather than the model, generating certified equivalent answer variants, that is, rewrites that preserve mathematical meaning by construction, so that any rejection is a provable false negative needing no human adjudication. We then measure rejection per answer category across four widely used verifiers over 307,420 verdicts. We find three things. (1) Self validation ranges from 53.8% to 95.2% on identical inputs, a spread of 41.3 points. The published figure describes one implementation, not the task; two configurations of the same library disagree on 49.9% of pairs. (2) The residual is not spread across parsing categories but concentrated in whitespace and punctuation, which account for 93.0% of in contract failures for the default LaTeX configuration. A trailing period or newline dominates the budget. (3) Separating rejection from execution failure shows that verifiers with similar aggregate error fail for opposite reasons, and that a reference numeric cascade accepts off by one wrong answers as a step function of magnitude, from 0% below 10^4 to 100% at or above, because its relative tolerance is scale invariant.
