Search papers, labs, and topics across Lattice.
This paper addresses the challenges of IoT firmware vulnerability detection by introducing IoTVulBench, a human-verified benchmark designed for cross-corpus evaluation. The study demonstrates that models trained on IoTVulBench significantly outperform existing datasets, achieving a maximum Matthews Correlation Coefficient (MCC) of 0.73 through staged curriculum learning and diversity-optimized ensembles. These findings highlight that the quality of training data and curriculum strategies are crucial for enhancing generalization in vulnerability detection, rather than merely increasing model size.
Models trained on IoTVulBench outperformed existing benchmarks by up to 0.42 MCC, showcasing the critical role of domain-specific training and curriculum design in vulnerability detection.
IoT firmware vulnerability detection remains challenging due to heterogeneous firmware ecosystems, resource-constrained platforms, and limitations in existing benchmarks. Many datasets are synthetic or general-purpose and lack human-verified, contamination-screened annotations, limiting evidence on cross-corpus generalization across training sources, model architectures, and curriculum strategies. To address this gap, this paper introduces IoTVulBench, a human-verified benchmark for cross-corpus firmware vulnerability detection. IoTVulBench-Core was constructed from GitHub repositories, validated by three expert reviewers, and evaluated on a contamination-screened held-out target across five model architectures, two tuning methods, and three curriculum strategies, with ensemble, distillation, and robustness analyses. Models trained on IoTVulBench achieved the highest MCC among matched single-source datasets, reaching 0.58 versus 0.44 for PrimeVul and 0.39 for D2A. Staged curriculum learning increased MCC to 0.69, while a diversity-optimized ensemble achieved 0.73, improving by 0.42 MCC over the strongest reference comparator, a static analyzer at 0.31, and by 0.29 over PrimeVul. At a 0.5% false-positive rate, the model missed only 21% of vulnerabilities, compared with 71% for the strongest comparator. It retained 86% of its performance under identifier renaming and demonstrated strong calibration and largely faithful explanations. These findings indicate that domain-matched training data and curriculum design, rather than model scale alone, are key drivers of generalization in firmware vulnerability detection. The results provide a benchmark for future research and deployment-ready configurations for practical IoT security applications.
