Search papers, labs, and topics across Lattice.
This paper compares the performance of LLMs (Llama 3.1, Mistral, Gemma 3, Phi-4) against a fine-tuned DistilBERT model for sarcasm detection in Hinglish code-mixed text. The study reveals that DistilBERT, fine-tuned on a small amount of LLM-generated code-mixed data, achieves a higher accuracy (84%) than the LLMs in zero-shot and few-shot settings. This suggests that domain-specific fine-tuning of smaller models can be more effective than relying on the general knowledge of LLMs in low-resource scenarios.
Fine-tuning a small DistilBERT model on LLM-generated data beats the pants off larger LLMs at code-mixed Hinglish sarcasm detection.
Sarcasm detection in multilingual and code-mixed environments remains a challenging task for natural language processing models due to structural variations, informal expressions, and low-resource linguistic availability. This study compares four large language models, Llama 3.1, Mistral, Gemma 3, and Phi-4, with a fine-tuned DistilBERT model for sarcasm detection in code-mixed Hinglish text. The results indicate that the smaller, sequentially fine-tuned DistilBERT model achieved the highest overall accuracy of 84%, outperforming all of the LLMs in zero and few-shot set ups, using minimal LLM generated code-mixed data used for fine-tuning. These findings indicate that domain-adaptive fine-tuning of smaller transformer based models may significantly improve sarcasm detection over general LLM inference, in low-resource and data scarce settings.
