Search papers, labs, and topics across Lattice.
This paper introduces GHR-VLM, a visual grounded hybrid reasoning framework designed for zero-shot analytics of transit bus video, addressing the limitations of existing supervised models and vision-language models (VLMs). By employing an edge-cloud architecture, the system continuously monitors door status and segments passenger clips, allowing the VLM to focus on localized evidence for identifying boarding passengers and classifying payment behaviors. Evaluation on extensive real-world bus surveillance footage reveals that GHR-VLM effectively enhances passenger-level analytics while navigating challenges associated with degraded video conditions.
Grounded hybrid reasoning can transform zero-shot transit video analytics, drastically improving the accuracy of passenger payment behavior classification without the need for extensive training data.
Transit video understanding can provide valuable fine-grained data that conventional passenger counters and fare systems cannot capture. However, supervised video models require task-specific annotations, while applying vision-language models (VLMs) directly to long onboard videos is unreliable and costly. To leverage the complementary strengths of both approaches, we propose GHR-VLM, a visual grounded hybrid reasoning framework for zero-shot transit-bus video analytics. It is motivated by the observation that explicit visual grounding can improve VLM reasoning by converting long surveillance streams into compact, passenger-centered spatiotemporal evidence. Specifically, we propose an edge-cloud design in which a lightweight edge-based monitor continuously tracks door status and segments passenger clips. A backend VLM then identifies boarding passengers and classifies payment behavior through a two-stage coarse-to-fine refinement of spatiotemporal evidence. By invoking the VLM only on grounded passenger clips and contact sheets, GHR-VLM reduces cloud inference, avoids payment-specific training data, and supplies the localized evidence that VLMs otherwise struggle to identify. Evaluation on 486 minutes of real-world bus surveillance video demonstrates the potential of grounded edge-cloud reasoning for passenger-level payment analytics while highlighting the challenges posed by degraded video conditions.
