PETRA: Transforming Web Text for Petroleum-Engineering Domain Adaptation

2026-06-23

AI Summary

This study addresses the challenge of scarce supervision signals in petroleum engineering retrieval, where abundant online texts lack domain-specific labels. The authors propose an integrated data construction strategy that combines high-recall domain filtering with a high-precision classifier (98.4% accuracy) to curate clean corpora, augmented by chunk-anchored query generation, LLM-synthesized hard negatives, and retrieval-based mining to build a high-quality, domain-adapted dataset. Experiments demonstrate significant improvements: nDCG in first-stage retrieval rises from 0.703 to 0.763, yielding a 44% relative gain on a public geoscience benchmark and an average 23% improvement across six reasoning-intensive tasks. Notably, the work reveals no direct correlation between synthetic-label training accuracy and final retrieval performance, offering new insights into weakly supervised retrieval.

Abstract

Petroleum-engineering search exposes a supervision gap for strong general retrievers: relevant evidence exists in public web text, but domain relevance labels are scarce. To address this gap, we propose PETRA, a large-scale Petroleum Engineering Text for Retrieval Adaptation dataset and pipeline that converts noisy public web data into a curated domain corpus and synthetic supervision for dense retrieval and reranking. PETRA contains 1.36M curated chunks, approximately 2B token equivalents, $\approx$859k, embedding training rows from $\approx$224k anchors, and roughly 400k teacher-scored reranker candidate rows. Its construction combines high-recall energy-domain curation, an energy-domain classifier with 98.4% test accuracy, chunk-grounded query generation, LLM-written hard negatives, and retrieval-mined candidate lists. PETRA improves first-stage in-domain Normalized Discounted Cumulative Gain (nDCG) from 0.703 to 0.763 through score fusion. Reranker adaptation improves the public Earth Science benchmark by 44% relative and a six-task reasoning-intensive panel by 23%. Failed training recipes show that high train-holdout accuracy on synthetic labels does not predict retrieval gains; retrieval-mined data helps only after being repackaged as teacher-scored candidate lists sampled from the inference-time candidate distribution.

Problem

• domain adaptation
• information retrieval
• supervision gap
• petroleum engineering
• web text

Innovation

• domain adaptation
• dense retrieval
• synthetic supervision
• hard negatives
• retrieval-mined data