Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a fine-tuned RoBERTa-base model for classifying CVE descriptions into CWE categories, trained on a large dataset of 234,770 AI-refined and filtered CVE-CWE pairs. The model achieves 87.4% top-1 accuracy and 60.7% Macro F1 on a held-out test set, significantly outperforming a TF-IDF baseline, especially on rare weakness categories. Remarkably, the 125M parameter RoBERTa model achieves performance statistically equivalent to an 8B parameter model on the CTI-Bench benchmark, highlighting its parameter efficiency.
A fine-tuned RoBERTa model with only 125M parameters can match the CVE-to-CWE classification accuracy of models 64x larger, proving that strategic fine-tuning and data curation can close the gap between small and large language models.
We present a fine-tuned RoBERTa-base classifier (125M parameters) for mapping Common Vulnerabilities and Exposures (CVE) descriptions to Common Weakness Enumeration (CWE) categories. We construct a large-scale training dataset of 234,770 CVE descriptions with AI-refined CWE labels using Claude Sonnet 4.6, and agreement-filtered evaluation sets where NVD and AI labels agree. On our held-out test set (27,780 samples, 205 CWE classes), the model achieves 87.4% top-1 accuracy and 60.7% Macro F1 -- a +15.5 percentage-point Macro F1 gain over a TF-IDF baseline that already reaches 84.9% top-1, demonstrating the model's advantage on rare weakness categories. On the external CTI-Bench benchmark (NeurIPS 2024), the model achieves 75.6% strict accuracy (95% CI: 72.8-78.2%) -- statistically indistinguishable from Cisco Foundation-Sec-8B-Reasoning (75.3%, 8B parameters) at 64x fewer parameters. We release the dataset, model, and training code.