Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces HuGLEN, an evaluation pipeline that leverages an LLM-as-a-judge alongside expert ratings to systematically compare and rank various large language models (LLMs) for optical network automation tasks. By applying this method to translate outputs from an explainable AI model for quality of transmission estimation, the authors find that a medium-sized LLM (12B parameters) achieves the highest quality efficiency score (QES), indicating optimal balance between explanation quality and computational efficiency. The approach not only minimizes the need for extensive human labeling but also enhances the consistency of model selection for practical applications in network automation.
A medium-sized LLM outperforms larger models in delivering high-quality, efficient explanations for optical network automation tasks.
Large language models (LLMs) are increasingly adopted for network automation, yet their output quality and inference cost can vary substantially across LLM families. We present HuGLEN, a stepwise evaluation pipeline that uses an LLM-as-a-judge together with a small set of expert ratings to enable scalable and reproducible comparison of candidate LLMs, and to rank them using a quality efficiency score (QES). We demonstrate HuGLEN for translating outputs from an explainable artificial intelligence (XAI) model for the optical network quality of transmission (QoT) estimation task into operator-friendly explanations. Our results show that a medium-sized LLM (12B parameters) achieves the highest QES, indicating the best trade-off between explanation quality and efficiency. Overall, HuGLEN reduces the human-labeling burden while supporting consistent model selection for operator-facing automation tasks.