Search papers, labs, and topics across Lattice.
This study leverages Large Language Models (LLMs) to automate the injection of vulnerabilities into Solidity smart contracts, addressing the challenge of scarce datasets for vulnerability detection. By validating the injected contracts through a comprehensive multi-step pipeline, the authors successfully generated nearly 1,000 candidate variants, with 32 confirmed vulnerable contracts across 25 vulnerability types. The findings highlight both the potential of LLMs in vulnerability injection and the limitations regarding scalability and diversity in the generated contracts.
LLMs can autonomously inject vulnerabilities into smart contracts, yielding a surprising 16.58% survival rate of confirmed vulnerabilities across diverse types.
Assessing vulnerability detection tools for smart contracts requires datasets with known ground truth, yet such datasets are scarce and difficult to build by hand. We propose an approach that uses Large Language Models (LLMs) to automatically inject vulnerabilities into Solidity smart contracts, and demonstrate it in a case study targeting 49 vulnerability types from OpenSCV. Injected contracts are validated through a multi-step pipeline checking compilation, execution, business logic, and the presence of the intended vulnerability. Applied to real-world contracts from SmartBugs, LLMs generate nearly 1,000 candidate variants; after deduplication and validation, 32 confirmed vulnerable contracts spanning 25 vulnerability types survive (a 16.58% survival rate). Surviving contracts concentrate in structurally simpler targets and vulnerability types with localized syntactic patterns. We report practical challenges including LLMs'non-determinism and the difficulty of preserving contract semantics. We then use the validated contracts to assess three static analyzers, revealing complementary and incomplete coverage profiles. Results show that LLM-based vulnerability injection is feasible, while exposing key limitations in scalability and diversity.
