GHR-VLM: Making Zero-Shot Transit Video Analytics Realizable with Grounded Hybrid Reasoning

2026-07-15

AI Summary

This study addresses the challenge of accurately recognizing passenger boarding, alighting, and payment behaviors in unlabeled public transit videos. Direct application of vision-language models (VLMs) to long videos suffers from low reliability and high computational cost. To overcome these limitations, the authors propose an edge–cloud collaborative hybrid reasoning framework that integrates explicit visual grounding with VLMs for the first time in a bus monitoring context. A lightweight edge module extracts passenger-centric spatiotemporal clips and detects door states, while the cloud-based VLM performs zero-shot behavior understanding through a two-stage coarse-to-fine inference process. The approach eliminates the need for payment behavior annotations and substantially reduces cloud-side computational overhead. Experiments on 486 minutes of real-world video demonstrate its effectiveness for passenger-level payment analysis and highlight the challenges posed by low-quality footage.

Abstract

Transit video understanding can provide valuable fine-grained data that conventional passenger counters and fare systems cannot capture. However, supervised video models require task-specific annotations, while applying vision-language models (VLMs) directly to long onboard videos is unreliable and costly. To leverage the complementary strengths of both approaches, we propose GHR-VLM, a visual grounded hybrid reasoning framework for zero-shot transit-bus video analytics. It is motivated by the observation that explicit visual grounding can improve VLM reasoning by converting long surveillance streams into compact, passenger-centered spatiotemporal evidence. Specifically, we propose an edge-cloud design in which a lightweight edge-based monitor continuously tracks door status and segments passenger clips. A backend VLM then identifies boarding passengers and classifies payment behavior through a two-stage coarse-to-fine refinement of spatiotemporal evidence. By invoking the VLM only on grounded passenger clips and contact sheets, GHR-VLM reduces cloud inference, avoids payment-specific training data, and supplies the localized evidence that VLMs otherwise struggle to identify. Evaluation on 486 minutes of real-world bus surveillance video demonstrates the potential of grounded edge-cloud reasoning for passenger-level payment analytics while highlighting the challenges posed by degraded video conditions.

Problem

• zero-shot video analytics
• transit video understanding
• vision-language models
• visual grounding
• passenger payment behavior

Innovation

• grounded reasoning
• edge-cloud architecture
• zero-shot video analytics
• vision-language models
• spatiotemporal evidence