Relational Probing: LM-to-Graph Adaptation for Financial Prediction

2026-04-11

AI Summary

This work addresses the high autoregressive decoding overhead and misalignment between graph construction and downstream tasks in existing prompt-based language models for financial entity relation extraction. The authors propose Relational Probing, a method that replaces the language model’s output head with a relation head to directly generate structured relation graphs end-to-end from hidden states, jointly trained with a stock trend prediction model. This approach achieves the first efficient induction of relation graphs from small language models (SLMs) while preserving semantic expressiveness and enforcing strict graph structural constraints. Experiments on the Qwen3 series (0.6B/1.7B/4B) demonstrate that the method trains efficiently on a single 24GB GPU, significantly outperforms co-occurrence baselines, and maintains controllable inference costs.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Graph-based Machine Learning
• Data Mining & Knowledge Management: Linked Open Data, Knowledge Graphs & KB Completion

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Language models can be used to identify relationships between financial entities in text. However, while structured output mechanisms exist, prompting-based pipelines still incur autoregressive decoding costs and decouple graph construction from downstream optimization. We propose \emph{Relational Probing}, which replaces the standard language-model head with a relation head that induces a relational graph directly from language-model hidden states and is trained jointly with the downstream task model for stock-trend prediction. This approach both learns semantic representations and preserves the strict structure of the induced relational graph. It enables language-model outputs to go beyond text, allowing them to be reshaped into task-specific formats for downstream models. To enhance reproducibility, we provide an operational definition of small language models (SLMs): models that can be fine-tuned end-to-end on a single 24GB GPU under specified batch-size and sequence-length settings. Experiments use Qwen3 backbones (0.6B/1.7B/4B) as upstream SLMs and compare against a co-occurrence baseline. Relational Probing yields consistent performance improvements at competitive inference cost.

Problem

• relational graph
• financial prediction
• language model adaptation
• structured output
• stock-trend prediction

Innovation

• Relational Probing
• language-model-to-graph adaptation
• structured relational graph
• joint training
• small language models