Search papers, labs, and topics across Lattice.
This paper addresses the challenge of text-based person anomaly retrieval by developing a heterogeneous vision-language ensemble framework that integrates multiple embedding models through score alignment and iterative fusion. The approach enhances retrieval accuracy by employing disagreement-aware reranking to refine results for ambiguous queries, leading to significant improvements in performance metrics. On the Pedestrian Anomaly Behavior benchmark, the method achieved a mean Average Precision (mAP) of 90.92% and high recall rates, underscoring the effectiveness of combining diverse vision-language representations for this complex task.
Achieving over 90% mAP in text-based person anomaly retrieval reveals the power of heterogeneous vision-language ensembles and selective multimodal reasoning.
Text-based person anomaly retrieval aims to retrieve pedestrians exhibiting anomalous behaviors from a large image gallery using natural language descriptions. Compared with conventional text-based person retrieval, this task requires fine-grained reasoning over pedestrian appearance, behaviors, object interactions, and scene context, making robust cross-modal matching significantly more challenging. This paper presents the GENAI4E team's solution to AI City Challenge 2026 Track 4. Our framework builds upon a strong retrieval backbone and progressively integrates heterogeneous vision-language embedding models through score alignment and iterative ensemble fusion, followed by disagreement-aware VLM reranking for ambiguous queries. On the official Pedestrian Anomaly Behavior (PAB) benchmark, our approach achieves 90.92% mAP, 85.13% Recall@1, 97.72% Recall@5, and 98.68% Recall@10, demonstrating the effectiveness of combining complementary vision-language representations with selective multimodal reasoning for large-scale text-based person anomaly retrieval.
