Search papers, labs, and topics across Lattice.
This paper introduces TreeAdapter, a framework that enhances fine-grained image generation of rare biological species by utilizing hierarchical taxonomic data. By employing lightweight adapters at each node of a taxonomic tree, TreeAdapter effectively captures both species-specific traits and shared visual features among related taxa, addressing the limitations of traditional models that either over-isolate species or fail to generalize. Extensive experiments reveal that TreeAdapter achieves state-of-the-art performance on biodiversity benchmarks, significantly outperforming existing general-purpose and domain-specific models in generating accurate visual representations.
TreeAdapter revolutionizes species image generation by harnessing hierarchical taxonomic structures, achieving unprecedented detail and accuracy in fine-grained outputs.
Although general text-to-image models excel in open-domain generation, their performance degrades significantly in specialized downstream domains, particularly when generating images of rare biological species. Hindered by long-tailed distributions, general models struggle to capture subtle fine-grained details, while per-species fine-tuning methods over-isolate individual species and consequently ignore the shared visual features among closely related taxa. To address this, we propose TreeAdapter, a novel framework that explicitly leverages hierarchical taxonomic data. Rather than using a monolithic model or independent per-species modules, TreeAdapter attaches lightweight adapters to every node of the taxonomic tree. Specifically, leaf-node adapters capture species-specific visual traits, while internal-node adapters encapsulate shared semantics among descendant taxa. We introduce a two-stage training paradigm where ancestor adapters are optimized to model only the residual visual features unexplained by their descendants. This model architecture and training paradigm enable the model to fully leverage hierarchical information, ensuring the accurate generation of visual features for each species. Extensive experiments across three large-scale biodiversity benchmarks demonstrate that TreeAdapter achieves state-of-the-art fine-grained generation quality, outperforming both general-purpose and domain-specific baselines.
DAMO