PHAGE: Patent Heterogeneous Attention-Guided Graph Encoder for Representation Learning

2026-05-11

AI Summary

This work addresses the limitation of existing patent encoding methods that linearize claims and thereby neglect their intrinsic directed dependency structure, resulting in the loss of hierarchical semantic information. To overcome this, the study proposes the first approach that explicitly models heterogeneous claim dependencies in patent representation learning by constructing a heterogeneous graph to distinguish legal citations from technical associations. The intra-document topological structure is incorporated as a strong inductive bias into a Transformer architecture through relation-aware attention biases, connectivity masks, and dual-granularity contrastive learning, which are jointly optimized. Experimental results demonstrate that the proposed method significantly outperforms current baselines across classification, retrieval, and clustering tasks, confirming both the effectiveness of modeling claim topology and its persistent representational capacity within learned model weights.

Abstract

Patent claims form a directed dependency structure in which dependent claims inherit and refine the scope of earlier claims; however, existing patent encoders linearize claims as text and discard this hierarchy. Directly encoding this structure into self-attention poses two challenges: claim dependencies mix relation types that differ in semantics and extraction reliability, and the dependency graph is defined over claims while Transformers attend over tokens. PHAGE addresses the first challenge through a deterministic graph construction pipeline that separates near-deterministic legal citations from noisier rule-based technical relations, preserving type distinctions as heterogeneous edges. It addresses the second through a connectivity mask and learnable relation-aware biases that lift claim-level topology into token-level attention, allowing the encoder to differentially weight each relation type. A dual-granularity contrastive objective then aligns representations with both inter-patent taxonomy and intra-patent topology. PHAGE outperforms all baselines on classification, retrieval, and clustering, showing that intra-document claim topology is a stronger inductive bias than inter-document structure and that this bias persists in the encoder weights after training.

Problem

• patent representation
• claim dependency
• heterogeneous graph
• structured attention
• hierarchical encoding

Innovation

• heterogeneous graph
• claim dependency
• relation-aware attention
• dual-granularity contrastive learning
• patent representation learning