Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study evaluates the limitations of traditional sentiment analysis in customer support by analyzing 70,450 conversations from an online fundraising platform using GPT-5.4 to assess customer satisfaction and identify reported issues. The results show that the satisfaction estimates correlate more closely with customer ratings than sentiment analysis, with a correlation of 0.47 compared to 0.36, and reveal that tone and satisfaction often diverge in nearly half of the cases. This approach highlights the prevalence of "tolerated friction," where customers are satisfied yet report unresolved problems, suggesting that LLM-based methods can provide deeper insights into customer experiences than sentiment analysis alone.
LLM-based analysis reveals that nearly half of customer interactions show a disconnect between sentiment and actual satisfaction, exposing critical unresolved issues that sentiment analysis misses.
Most companies read their customer support data at scale using sentiment analysis, which measures how customers sound rather than whether they were satisfied with the result. We tested a richer alternative on 70,450 support conversations from a leading online fundraising platform: alongside tone, we used GPT-5.4 to estimate each customer's satisfaction and to flag whether they reported a concrete problem, then validated all three readings against the 1-to-5 ratings customers left on the conversations they rated. The satisfaction estimate tracked those ratings far better than sentiment did, correlating at 0.47 against 0.36 and flagging unhappy customers with far fewer false alarms. The structured read also sees what sentiment cannot: tone and satisfaction disagree in 44% of conversations, a single "Neutral" label hides everything from quietly satisfied customers to ones who quietly gave up, and the largest group of all is "tolerated friction," customers who are satisfied but still reporting a fixable problem, a standing issue that no sentiment-based dashboard can surface. The broader finding is that LLM-based annotation can capture far more than the tonality of a customer's language, offering strong potential for new business metrics grounded instead in the customer's state (whether they were satisfied) and the cause of their problem extracted directly from the raw textual data of interactions and feedback.