Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces an output-only black-box attack on genomic language models (GLMs) to extract training data by exploiting convergent repeated generations. The attack successfully recovers significant segments of training sequences from Evo 2 7B, demonstrating that convergence in generation is a practical signal for data extraction without requiring gradient information or token likelihoods. The findings characterize the influence of various generation parameters on the success rate of these data extraction attacks.
Genomic language models can leak substantial training data segments through simple repeated generation, even without access to gradients or token probabilities.
Genomic language models generate long nucleotide continuations from short prefixes, exposing a direct route for studying training-sequence extraction through the generation interface. We present an output-only black-box attack on Evo 2 that selects convergent groups of repeated generations without gradients, token likelihoods, or training-corpus access during candidate selection. Recovered content is then scored by an evaluator combining aligned length, nucleotide identity, match multiplicity, and sequence complexity. Across 304 runs on Evo 2 7B with bacteriophage and plasmid prefixes, the attack yields 888 candidate clusters, of which 85 match training segments of at least 500 bp, giving a cluster-level attack success rate of 9.57%. Final filtering retains 22 uniquely matched extraction events at the 500-bp threshold and 15 at 1,000 bp, covering 14 and 7 distinct training-sequence regions, respectively. Within-category source-homology screening retains 11 of the 22 events at a 99% exclusion threshold and four at 98%, with the latter mapping to two reference regions. The results establish convergence among repeated nucleotide generations as a practical output-only signal for recovering long training-sequence segments and characterize how recovery depends on prefix length, cluster size, perplexity, early stopping, and sequence domains.