🤖 AI Summary
Existing causal discovery datasets predominantly focus on explicit, direct causal relationships and struggle to capture the implicit, nested, and higher-order causal structures commonly found in climate reports. To address this gap, this work introduces ClimateCause—a novel, expert-annotated dataset of climate policy texts—where implicit causal statements are normalized and decomposed into atomic causal triplets, enabling the construction of causal graphs that integrate causal relevance, relation types, and spatiotemporal context. Leveraging this dataset, we propose a readability quantification method based on the semantic complexity of causal graphs and establish a benchmark for evaluating large language models (LLMs) on complex causal reasoning tasks. Experiments demonstrate that ClimateCause effectively supports causal chain inference and relevance prediction, while also revealing critical limitations of current LLMs in handling higher-order causal structures.
📝 Abstract
Understanding climate change requires reasoning over complex causal networks. Yet, existing causal discovery datasets predominantly capture explicit, direct causal relations. We introduce ClimateCause, a manually expert-annotated dataset of higher-order causal structures from science-for-policy climate reports, including implicit and nested causality. Cause-effect expressions are normalized and disentangled into individual causal relations to facilitate graph construction, with unique annotations for cause-effect correlation, relation type, and spatiotemporal context. We further demonstrate ClimateCause's value for quantifying readability based on the semantic complexity of causal graphs underlying a statement. Finally, large language model benchmarking on correlation inference and causal chain reasoning highlights the latter as a key challenge.