OmniClimate-TC: Physics-Aware Visual Abstractions for Multimedia Reasoning over Tropical Cyclones

2026-08-30

AI Summary

本文提出了一种物理感知视觉抽象方法PAVA，用于解决视觉-语言模型在处理热带气旋等极端天气时面临的挑战，并基于此构建了OmniClimate-TC基准测试。

Abstract

Meteorological reanalysis encodes extreme weather through continuous, physically constrained fields, posing a fundamental challenge for vision-language models (VLMs) whose perceptual assumptions are shaped by natural images. Tropical cyclones exemplify this mismatch: critical properties such as intensity extrema, asymmetry, spatial extent, and physical impacts arise from field-level organization rather than object-centric visual cues. Existing approaches address this gap through text alignment or annotation, treating the problem as multimodal supervision rather than representation design. We introduce Physics-Aware Visual Abstraction (PAVA), a plug-and-play physics-aware representation and annotation interface that maps physical reanalysis fields to visually identifiable and semantically grounded perceptual abstractions for supervision and evaluation in vision-language reasoning. Building on PAVA, we construct OmniClimate-TC, a benchmark for tropical cyclone analysis spanning five classes of reasoning and nine tasks, with 243,890 physically grounded instruction-tuning pairs. Using PAVA-aligned supervision, we adapt VLMs and provide evidence that this representation design improves reasoning over tropical cyclone hazard fields. Our results position OmniClimate-TC as a benchmark for multimedia reasoning over structured geophysical fields, and highlight representation design as a key ingredient for physically grounded reasoning in scientific media.

Problem

• Tropical Cyclones
• Vision-Language Models
• Meteorological Reanalysis
• Physical Fields
• Perceptual Abstractions

Innovation

• Physics-Aware Visual Abstraction
• Tropical Cyclones
• Vision-Language Models
• Multimedia Reasoning
• Physical Reanalysis Fields