Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates how the tone of prompts influences both the accuracy of responses and the inference cost in large language models (LLMs), utilizing a 570-question MMLU dataset across seven tonal variations. The findings reveal that variations in output token length due to tone changes can exceed accuracy variations by a significant margin, with consumption fluctuating by up to 44.3%. Notably, the analysis indicates that certain tones, particularly rude and neutral, dominate the Pareto-optimal frontier in specific model architectures, highlighting a critical interplay between response quality and resource expenditure.
Prompt tone can drastically alter both the accuracy of LLM responses and the inference costs, with output token consumption varying by over 44% across different tones.
We examine how prompt tone affects both accuracy of the LLM answers and inference cost as reflected in output-token consumption. Experiments were performed to understand the trade-offs between accuracy and inference cost on a 570 Question MMLU dataset for LLM models prompted in seven different tones from sycophantic to threatening. Our results show that the output-token-length variation substantially exceeded accuracy variation across all models. Output-token consumption varied by up to 44.3% across tone conditions. We also analyzed the tradeoff between the accuracy of the answers and the average output token length in the reasoning process. For the ChatGPT models 4o and 5-nano, the rude tone is quite dominant. For the Gemini models 2.5 Flash and 2.5 Flash Lite, the rude and neutral tones are dominant on the Pareto-optimal frontier. We find that prompt tone influences not only answer quality but also the amount of billable inference resources consumed by modern LLMs.