Search papers, labs, and topics across Lattice.
The paper introduces DANCE, a doubly adaptive nearest-neighbor conformal prediction algorithm for improved uncertainty quantification in deep learning classification. DANCE leverages a task-adaptive kernel regression model fit on the embedding layer to learn a kernel space tailored to the target task. This approach yields more efficient and robust prediction sets compared to existing logit-based and zero-shot conformal methods, as demonstrated across various datasets.
DANCE shrinks prediction sets by learning a task-specific kernel directly from the data's embeddings, offering a more efficient and robust approach to conformal prediction than relying solely on logit scores.
The recent developments of complex deep learning models have led to unprecedented ability to accurately predict across multiple data representation types. Conformal prediction for uncertainty quantification of these models has risen in popularity, providing adaptive, statistically-valid prediction sets. For classification tasks, conformal methods have typically focused on utilizing logit scores. For pre-trained models, however, this can result in inefficient, overly conservative set sizes when not calibrated towards the target task. We propose DANCE, a doubly locally adaptive nearest-neighbor based conformal algorithm combining two novel nonconformity scores directly using the data's embedded representation. DANCE first fits a task-adaptive kernel regression model from the embedding layer before using the learned kernel space to produce the final prediction sets for uncertainty quantification. We test against state-of-the-art local, task-adapted and zero-shot conformal baselines, demonstrating DANCE's superior blend of set size efficiency and robustness across various datasets.
