Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study addresses the challenge of AI-generated text detection, particularly the drop in performance when models encounter out-of-distribution (OOD) texts. By employing Bayesian data mixing and fine-tuning BERT-tiny models with Bayesian classification heads, the authors create a consolidated training set from three diverse datasets, leading to improved generalization. The results show that the MCGrad model achieves the highest mean score of 0.974 on the PAN 2026 test set, demonstrating that strategic dataset curation significantly enhances OOD performance.
Careful dataset curation can elevate OOD performance in AI text detection, with the MCGrad model achieving a leaderboard-topping score of 0.974.
Existing research shows that AI-generated text detection classifiers achieve strong in-distribution (ID) performance but do not maintain the same performance on out-of-distribution (OOD) texts, suggesting overfitting to dataset-specific features. However, combining different training datasets doesn't always improve performance and, in some cases, can even encourage shortcut learning. To address this issue, we fine-tune BERT-tiny models with Bayesian classification heads to select texts across three different datasets to use as a consolidated training set. We trained three different classifiers: fine-tuned DeBERTa-V3-large and ModernBERT-large classifiers via empirical X-risk minimization, and an MCGrad model that calibrates the predictions from the ModernBERT-large classifier. The DeBERTa-V3-large-large classifier achieves a mean score of 0.882 on the PAN 2026 test set across five metrics: AUROC, $F_1$, C@1, Brier score, and $F_{0.5u}$. ModernBERT-large achieves a score of 0.96 while MCGrad achieves the best score of the three with a mean score of 0.974, ranking second on the leaderboard. Our results highlight that careful dataset curation can lead to strong OOD performance. We release our ModernBERT-large and DeBERTa-V3-large models at https://huggingface.co/collections/ShantanuT01/panclef-2026 .