Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Adaptive Resolution Label Aggregation (ARLA), a novel method designed to evaluate deep learning segmentation models under conditions of unreliable, noisy, and inconsistent labels. By dynamically adjusting the resolution of both labels and model predictions during inference, ARLA enhances the analysis of model performance, particularly in challenging scenarios such as flood prediction where label inconsistencies are prevalent. The results demonstrate that ARLA significantly improves the evaluation of model behavior, effectively mitigating the impact of label noise and providing a clearer signal of true performance.
ARLA reveals that adjusting label resolution can dramatically enhance model evaluation accuracy in the presence of noisy and inconsistent labels.
Labels are critical for both training and evaluating deep learning segmentation models, but are often inconsistent, noisy, or ambiguous at class boundaries. Many approaches have been developed to support training models on weak labels, but few to none currently exist to facilitate evaluating models on unreliable labels. We therefore introduce a method called "Adaptive Resolution Label Aggregation", or "ARLA", which dynamically adapts the resolution of both the label and the model prediction at inference time before the evaluation metrics are computed. We demonstrate how ARLA can be used to better analyse model behaviour with a practical application to a real flood prediction model, where ARLA was able to overcome issues with inconsistent labelling of forested areas and errors in labels within regions of heavy cloud cover. Our work presents a new approach to evaluating segmentation models, with adjustable parameters to adapt the aggregated resolution to the precision of the label or the level of label noise. Fundamentally, ARLA exploits the information encapsulated by a label but minimises the label error, extracting from the noise a clearer signal of a model's true performance.