Search papers, labs, and topics across Lattice.
This paper investigates the inference-time energy consumption of LLM-based climate analysis chatbots, comparing domain-specific RAG systems (ChatNetZero and ChatNDC) to generic LLMs (GPT-4o-mini) under real-world user queries. The study decomposes RAG workflows into retrieval, generation, and hallucination-checking components to estimate energy use across different times of day and geographic locations. Results indicate that more agentic RAG pipelines significantly increase energy consumption, particularly due to accuracy and verification checks, without necessarily improving response quality proportionally.
Domain-specific LLM applications can consume surprisingly more energy than generic LLMs, especially when designed with complex, agentic RAG pipelines for enhanced accuracy.
As large language models (LLMs) are increasingly used in domain-specific applications, including climate change and environmental research, understanding their energy footprint has become an important concern. The growing adoption of retrieval-augmented (RAG) systems for climate-domain specific analysis raises a key question: how does the energy consumption of domain-specific RAG workflows compare with that of direct generic LLM usage? Prior research has focused on standalone model calls or coarse token-based estimates, while leaving the energy implications of deployed application workflows insufficiently understood. In this paper, we assess the inference-time energy consumption of two LLM-based climate analysis chatbots (ChatNetZero and ChatNDC) compared to the generic GPT-4o-mini model. We estimate energy use under actual user queries by decomposing each workflow into retrieval, generation, and hallucination-checking components. We also test across different times of day and geographic access locations. Our results show that the energy consumption of domain-specific RAG systems depends strongly on their design. More agentic pipelines substantially increase inference-time energy use, particularly when used for additional accuracy or verification checks, although they may not yield proportional gains in response quality. While more research is needed to further test these initial findings more robustly across models, environments and prompting structures, this study provides a new understanding on how the design of domain-specific LLM products affects both the energy footprint and quality of output.
