Track genomics, neuroscience and molecular biology.
Search papers, labs, and topics across Lattice.
This paper details an output-only black-box attack on genomic language models (GLMs) to extract training data by exploiting convergent repeated generations. The attack, applied to Evo 2 7B using bacteriophage and plasmid prefixes, successfully recovered 22 training segments of at least 500 bp and 15 segments of at least 1,000 bp. The findings demonstrate the practical feasibility of recovering long training sequences from GLMs and characterize factors influencing extraction success.
Genomic language models leak substantial training data through repeated generations, with a black-box attack recovering over 500 bp segments in nearly 10% of trials.
Genomic language models generate long nucleotide continuations from short prefixes, exposing a direct route for studying training-sequence extraction through the generation interface. We present an output-only black-box attack on Evo 2 that selects convergent groups of repeated generations without gradients, token likelihoods, or training-corpus access during candidate selection. Recovered content is then scored by an evaluator combining aligned length, nucleotide identity, match multiplicity, and sequence complexity. Across 304 runs on Evo 2 7B with bacteriophage and plasmid prefixes, the attack yields 888 candidate clusters, of which 85 match training segments of at least 500 bp, giving a cluster-level attack success rate of 9.57%. Final filtering retains 22 uniquely matched extraction events at the 500-bp threshold and 15 at 1,000 bp, covering 14 and 7 distinct training-sequence regions, respectively. Within-category source-homology screening retains 11 of the 22 events at a 99% exclusion threshold and four at 98%, with the latter mapping to two reference regions. The results establish convergence among repeated nucleotide generations as a practical output-only signal for recovering long training-sequence segments and characterize how recovery depends on prefix length, cluster size, perplexity, early stopping, and sequence domains.