POTracker: Optimizing Large Language Models for Standard-Compliant Power Outage Report Generation

2026-06-22

AI Summary

This study addresses the challenge of simultaneously ensuring semantic correctness and structural compliance when large language models generate power outage reports. To this end, the authors propose POTrackerLoss, a novel loss function that jointly optimizes textual semantics and structural tag similarity during fine-tuning of the Qwen2.5-7B-Instruct model, while incorporating explicit JSON/XML format constraints. This approach represents the first effort to co-optimize semantic accuracy and structural adherence in domain-specific report generation. Evaluated on a dataset of 1,000 real-world outage reports, the method achieves a structural accuracy of 86.47%, improves overall generation quality by 51% relative to baseline methods, and receives an expert rating of 4.03 out of 5, demonstrating significant performance gains.

Abstract

Recent large language models (LLMs) are good at general text generation, but it is still hard to use them for domain-specific data generation because the output must follow strict formatting and structural rules. Unlike open-ended tasks such as question answering or translation, domain-specific generation must be both semantically correct and compliant with existing guidelines and standards. In this work, we study the nationwide interoperability problem of utility power outage reports in the United States. In practice, outage reports need to be machine-readable (e.g., JSON or XML) and must strictly follow requirements from energy-sector regulatory bodies. To address this problem, we propose POTracker, an optimized LLM for power outage report generation. We fine-tune Qwen2.5-7B-Instruct using our proposed objective. The key contribution is a new loss function, POTrackerLoss, that considers both textual similarity and structural (tag) similarity between the generated report and the ground-truth report. We evaluate POTracker on a dataset of 1,000 power outage reports and compare it with five well-known fine-tuning methods and one rule-based XML conversion method. Results show that POTracker outperforms other fine-tuning approaches, improving overall accuracy by up to 51% and reaching 86.47% structural accuracy for generated power outage reports. In addition, we conduct a human study to assess the quality of the ground-truth standard reports, where domain experts assign the generated labels an average score of 4.03 on a 0--5 scale.

Problem

• power outage report
• standard compliance
• structured text generation
• interoperability
• domain-specific generation

Innovation

• POTracker
• structured text generation
• domain-specific LLM fine-tuning
• POTrackerLoss
• power outage reporting