Temporal Relation Extraction in Clinical Texts: A Span-based Graph Transformer Approach

2025-03-23

AI Summary

Addressing challenges in clinical temporal relation extraction—including long-document modeling, domain-specific terminology complexity, and sparse annotations—this paper proposes a novel method integrating span-based identification, a clinical-domain large language model (LPLM), and a heterogeneous graph transformer (HGT). The core innovation is a global landmark mechanism that explicitly captures long-range temporal dependencies among distant events. Our span-based framework jointly extracts clinical events and their temporal relations, while HGT performs unified reasoning over heterogeneous nodes—namely, events, temporal expressions, and contextual spans. Evaluated on the i2b2 2012 dataset, our approach achieves a new state-of-the-art TempEval F1 score, improving by 5.5 percentage points; notably, F1 for long-range temporal relations increases by 8.9%. These gains significantly enhance interpretability and practical utility in clinical decision support, diagnostic reasoning, and prognostic modeling.

Technology Category

• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data
• Planning, Routing, and Scheduling: Temporal Planning

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Temporal information extraction from unstructured text is essential for contextualizing events and deriving actionable insights, particularly in the medical domain. We address the task of extracting clinical events and their temporal relations using the well-studied I2B2 2012 Temporal Relations Challenge corpus. This task is inherently challenging due to complex clinical language, long documents, and sparse annotations. We introduce GRAPHTREX, a novel method integrating span-based entity-relation extraction, clinical large pre-trained language models (LPLMs), and Heterogeneous Graph Transformers (HGT) to capture local and global dependencies. Our HGT component facilitates information propagation across the document through innovative global landmarks that bridge distant entities. Our method improves the state-of-the-art with 5.5% improvement in the tempeval $F_1$ score over the previous best and up to 8.9% improvement on long-range relations, which presents a formidable challenge. This work not only advances temporal information extraction but also lays the groundwork for improved diagnostic and prognostic models through enhanced temporal reasoning.

Problem

• Extracting clinical events and temporal relations from text
• Handling complex clinical language and sparse annotations
• Improving long-range temporal relation extraction accuracy

Innovation

• Integrates span-based extraction and Graph Transformers
• Uses clinical large pre-trained language models
• Introduces global landmarks for distant entity relations