Search papers, labs, and topics across Lattice.
This study introduces POTracker, an optimized large language model specifically designed for generating power outage reports that comply with strict regulatory standards. By fine-tuning the Qwen2.5-7B-Instruct model with a novel loss function, POTrackerLoss, which balances textual and structural accuracy, the authors significantly enhance the model's ability to produce machine-readable reports. The results demonstrate a remarkable improvement in accuracy, achieving up to 51% better performance compared to existing methods and an impressive structural accuracy of 86.47%.
POTracker achieves a 51% boost in accuracy for generating compliant power outage reports, setting a new standard for domain-specific LLM applications.
Recent large language models (LLMs) are good at general text generation, but it is still hard to use them for domain-specific data generation because the output must follow strict formatting and structural rules. Unlike open-ended tasks such as question answering or translation, domain-specific generation must be both semantically correct and compliant with existing guidelines and standards. In this work, we study the nationwide interoperability problem of utility power outage reports in the United States. In practice, outage reports need to be machine-readable (e.g., JSON or XML) and must strictly follow requirements from energy-sector regulatory bodies. To address this problem, we propose POTracker, an optimized LLM for power outage report generation. We fine-tune Qwen2.5-7B-Instruct using our proposed objective. The key contribution is a new loss function, POTrackerLoss, that considers both textual similarity and structural (tag) similarity between the generated report and the ground-truth report. We evaluate POTracker on a dataset of 1,000 power outage reports and compare it with five well-known fine-tuning methods and one rule-based XML conversion method. Results show that POTracker outperforms other fine-tuning approaches, improving overall accuracy by up to 51% and reaching 86.47% structural accuracy for generated power outage reports. In addition, we conduct a human study to assess the quality of the ground-truth standard reports, where domain experts assign the generated labels an average score of 4.03 on a 0--5 scale.
