Search papers, labs, and topics across Lattice.
This paper introduces CHARM, a novel framework for sarcasm detection that leverages both text and acoustic cues without requiring fine-tuning of existing models. By employing Bidirectional Charge Calibration (BiCAL) to balance sarcastic and literal interpretations and Acoustic Late-Fusion Rescue (ALFR) to integrate prosodic features, the approach achieves significant improvements in sarcasm detection accuracy across multiple languages. The results show that BiCAL reaches a Macro-F1 score of 0.787 on MUStARD, while ALFR enhances weaker models by up to 0.382 Macro-F1 on CMMA, highlighting the effectiveness of multimodal integration in this context.
Zero-shot LLMs can be significantly improved for sarcasm detection by integrating acoustic cues, achieving record-breaking accuracy without any model fine-tuning.
Sarcasm detection, the identification of discrepancies between literal and intended meaning, is a fundamental task in affective computing. However, zero-shot instruction-tuned Large Language Models (LLMs) systematically over-predict the positive (sarcastic) class across the entire capability spectrum, while the prosodic cues humans rely on remain underexploited and transfer unevenly across languages. We introduce CHARM (Charge Calibration and Acoustic Rescue for Multimodal Sarcasm Detection), a training-free framework that couples two modules. Bidirectional Charge Calibration (BiCAL) steers the LLM toward opposing sarcastic and literal verdicts along a symmetric axis of charged prompts; the induced directional biases cancel by construction, and a simple aggregation recovers an unbiased pragmatic signal. Acoustic Late-Fusion Rescue (ALFR) then fuses the calibrated votes with prosodic descriptors and LLM-generated auditory-perception probes through a shallow classifier, actively down-weighting saturated text votes in favour of acoustic evidence. Without fine-tuning any backbone, BiCAL attains the highest reported zero-shot text-only Macro-F1 of 0.787 on MUStARD, while ALFR lifts weak backbones by up to +0.382 Macro-F1 on CMMA. A Stouffer meta-analysis confirms statistical significance on MUStARD and CMMA (Z = 13.89 and Z = 34.64, respectively; p<10^-43). Our analysis further uncovers a cross-cultural prosodic decoupling: low-level acoustics fail to transfer across languages, whereas high-level perceptual abstractions remain robust. Together, these components yield an explainable, cross-lingual multimodal detector.
