Search papers, labs, and topics across Lattice.
The paper introduces UniTraffic-Agent, a system designed to enhance traffic video understanding by employing a structured observe-reason-act-verify workflow to analyze sparse events and varied viewpoints in traffic videos. This approach addresses the challenges faced by multimodal large language models (MLLMs) in explaining traffic events, including their development, causation, and timing. The results demonstrate competitive performance on the AI City Challenge leaderboards, achieving notable rankings in Traffic Anomaly Reasoning, fisheye traffic event analysis, and pedestrian intention reasoning tasks.
UniTraffic-Agent achieves top-tier performance in complex traffic video reasoning tasks, showcasing the potential of structured workflows in multimodal AI applications.
Traffic video understanding has become an important problem in intelligent transportation, as road videos provide direct evidence for accidents, violations, and interactions between vehicles and vulnerable road users. A useful system should explain how a traffic event develops, why it happens, and when the relevant interaction occurs, yet this remains difficult for multimodal large language models (MLLMs) because traffic videos contain sparse events and varied viewpoints. We introduce UniTraffic-Agent, the MR-CAS solution for Track~3 of the 10th AI City Challenge, which includes Traffic Anomaly Reasoning (TAR) and two out-of-domain evaluations: FETV for fisheye traffic events and PSI-VQA for pedestrian intention reasoning. UniTraffic-Agent follows an observe--reason--act--verify workflow that samples timestamped visual evidence, reasons over all questions from the same clip in one request, and converts responses through task-specific action adapters. On the official Public leaderboards, MR-CAS ranks 16th on TAR with a score of 0.5780, 2nd on FETV with 0.4884, and 4th on PSI-VQA with 64.4161. The code is available at https://github.com/Roclp/UniTraffic-Agent.
