WeatherSyn: An Instruction Tuning MLLM For Weather Forecasting Report Generation

2026-05-08

AI Summary

This work proposes an automated weather forecast report generation task to address the inefficiency and information overload inherent in current practices that rely on manual integration of multi-source meteorological data. The authors construct the first instruction-tuning dataset for this purpose, encompassing 31 U.S. cities and eight weather variables, and train a specialized multimodal large language model on it. This model represents the first application of instruction tuning to weather report generation and demonstrates strong zero-shot generalization across geographic regions. Experimental results show that it outperforms leading closed-source multimodal large language models across multiple evaluation metrics, with particularly notable performance in describing complex weather phenomena and transferring knowledge to unseen regions.

Abstract

Accurate weather forecast reporting enables individuals and communities to better plan daily activities and agricultural operations. However, the current reporting process primarily relies on manual analysis of multi-source data, which leads to information overload and reduced efficiency. With the development of multimodal large language models (MLLMs), leveraging data-driven models to analyze and generate reports in the weather forecasting domain remains largely underexplored. In this work, we propose the Weather Forecasting Report (WFR) task and construct the first instruction-tuning dataset for this task, named~\DatasetNameL, which covers 31 cities in America and 8 weather aspects. Based on this corpus, we develop the first model, \ModelNameL, specialized in generating weather forecast reports. Evaluation across multiple metrics on our dataset shows that \ModelNameL~ consistently outperforms leading closed-source MLLMs, particularly on structurally complex weather aspects. We further analyze its performance across diverse geographic regions and weather aspects. \ModelNameL~ demonstrates strong transferability across different regions, highlighting its zero-shot generalization capability. \ModelNameL~offers valuable insight for developing MLLMs specialized in weather report generation. .

Problem

• weather forecasting
• report generation
• multimodal large language models
• instruction tuning
• data-driven modeling

Innovation

• instruction tuning
• multimodal large language model
• weather forecasting report generation
• zero-shot generalization
• domain-specific MLLM