Adverse-to-the-eXtreme Panoptic Segmentation: URVIS 2026 Study and Benchmark

2026-04-18

AI Summary

This work addresses the challenge of robust panoptic segmentation under extreme adverse weather conditions by introducing URVIS 2026, the first multimodal benchmark built upon the MUSES dataset that integrates RGB, LiDAR, radar, and event camera data. To enable consistent performance evaluation across diverse weather scenarios, the authors propose a weighted Panoptic Quality (wPQ) metric. The associated challenge attracted 17 participating teams with 47 submissions, from which four finalists advanced to the final round. Comprehensive evaluations reveal critical performance bottlenecks of current methods in extreme environments and highlight the untapped potential of multimodal fusion, thereby establishing a new benchmark and providing clear directions for advancing robust perception systems.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

This paper presents the report of the URVIS 2026 challenge on adverse-to-extreme panoptic segmentation. As the first challenge of its kind, it attracted 17 registered participants and 47 submissions, with 4 teams reaching the final phase. The challenge is based on the MUSES dataset, a multi-sensor benchmark for panoptic segmentation in adverse-to-extreme weather, including RGB frame camera, LiDAR, radar, and event camera data. Weighted Panoptic Quality (wPQ) is designed and adopted as the official ranking metric for fair evaluation across weather conditions. In this report, we summarise the challenge setting and benchmark results, analyse the performance of the submitted methods, and discuss current progress and remaining challenges for robust multimodal panoptic segmentation. Link: https://urvis-workshop.github.io/challenge-Muses.html

Problem

• panoptic segmentation
• adverse weather
• extreme conditions
• multimodal perception
• autonomous driving

Innovation

• adverse-to-extreme weather
• panoptic segmentation
• multimodal sensing
• Weighted Panoptic Quality
• MUSES dataset