The 9th AI City Challenge

2025-08-19

AI Summary

The 9th AI City Challenge addresses key urban domains—traffic management, industrial automation, and public safety—through four tasks: (1) multi-class 3D multi-camera tracking (persons, humanoid robots, AMRs, forklifts); (2) video question answering in traffic scenes using 3D gaze annotations; (3) spatial fine-grained reasoning in dynamic warehouses via RGB-D perception–language fusion; and (4) lightweight fisheye road-object detection for edge devices. Methodologically, it introduces novel 3D gaze labeling to enhance cross-camera event understanding, leverages NVIDIA Omniverse to generate high-fidelity synthetic RGB-D datasets, and integrates multi-camera calibration, 3D bounding-box annotation, model lightweighting, and multimodal language modeling. The challenge attracted 245 teams from 15 countries, with dataset downloads exceeding 30,000. Multiple tasks achieved new state-of-the-art results, significantly improving method reproducibility, cross-scenario generalizability, and edge-deployment efficiency.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Application Domains: Internet of Things, Sensor Networks & Smart Cities

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

The ninth AI City Challenge continues to advance real-world applications of computer vision and AI in transportation, industrial automation, and public safety. The 2025 edition featured four tracks and saw a 17% increase in participation, with 245 teams from 15 countries registered on the evaluation server. Public release of challenge datasets led to over 30,000 downloads to date. Track 1 focused on multi-class 3D multi-camera tracking, involving people, humanoids, autonomous mobile robots, and forklifts, using detailed calibration and 3D bounding box annotations. Track 2 tackled video question answering in traffic safety, with multi-camera incident understanding enriched by 3D gaze labels. Track 3 addressed fine-grained spatial reasoning in dynamic warehouse environments, requiring AI systems to interpret RGB-D inputs and answer spatial questions that combine perception, geometry, and language. Both Track 1 and Track 3 datasets were generated in NVIDIA Omniverse. Track 4 emphasized efficient road object detection from fisheye cameras, supporting lightweight, real-time deployment on edge devices. The evaluation framework enforced submission limits and used a partially held-out test set to ensure fair benchmarking. Final rankings were revealed after the competition concluded, fostering reproducibility and mitigating overfitting. Several teams achieved top-tier results, setting new benchmarks in multiple tasks.

Problem

• Multi-class 3D multi-camera tracking across diverse agents
• Video question answering with 3D gaze incident understanding
• Fine-grained spatial reasoning in dynamic warehouse environments

Innovation

• Multi-class 3D multi-camera tracking with detailed calibration
• Video question answering with 3D gaze labels
• RGB-D spatial reasoning in dynamic warehouse environments