Search papers, labs, and topics across Lattice.
DeepTaxon is introduced as a retrieval-augmented multimodal framework that unifies species identification and discovery by performing chain-of-thought comparative reasoning over retrieved visual evidence. It reframes species discovery as a retrieval-based decision problem, where novelty is determined by the sufficiency of evidence within the retrieval index, thus providing automatic supervision for both identification and discovery tasks. Trained via supervised fine-tuning and reinforcement learning, DeepTaxon demonstrates consistent improvements in both identification and discovery across various datasets, showcasing its scalability and zero-shot transfer capabilities.
Species identification and discovery, traditionally treated as separate problems, can be unified into a single framework that leverages retrieval-augmented reasoning for improved accuracy and interpretability.
Identifying species in biology among tens of thousands of visually similar taxa while discovering unknown species in open-world environments remains a fundamental challenge in biodiversity research. Current methods treat identification and discovery as separate problems, with classification models assuming closed sets and discovery relying on threshold-based rejection. Here we present DeepTaxon, a retrieval-augmented multimodal framework that unifies species identification and discovery through interpretable reasoning over retrieved visual evidence. Given a query image, DeepTaxon retrieves the top-$k$ candidate species with $n$ exemplar images each from a retrieval index and performs chain-of-thought comparative reasoning. Critically, we redefine discovery as an explicit, retrieval-based decision problem rather than an implicit parametric memory problem. A sample is novel if and only if the retrieval index lacks sufficient evidence for identification, so each retrieval naturally yields a classification or discovery label without manual annotation, thereby providing automatic supervision for both tasks. We train the framework via supervised fine-tuning on synthetic retrieval-augmented data, followed by reinforcement learning on hard samples, converting high-recall retrieval into high-precision decisions that scale to massive taxonomic vocabularies. Extensive experiments on a large-scale in-distribution benchmark and six out-of-distribution datasets demonstrate consistent improvements in both identification and discovery. Ablation studies further reveal effective test-time scaling with candidate count $k$ and exemplar count $n$, strong zero-shot transfer to unseen domains, and consistent performance across retrieval encoders, establishing an interpretable solution for biodiversity research.
