Search papers, labs, and topics across Lattice.
The paper introduces a unified benchmark dataset comprising 49 categories of human emotions, offensive content, and biases to evaluate LLM moderators. They fine-tuned Phi-4 using QLoRA, creating SafePhi, and demonstrated its superior performance compared to OpenAI Moderator and Llama Guard on the new benchmark. The study underscores the limitations of current LLM moderators and advocates for a human-in-the-loop approach to improve robustness and explainability.
Fine-tuning Phi-3 with QLoRA beats OpenAI and Meta's best moderation models, but still struggles with nuanced moral reasoning, proving AI safety is far from solved.
As AI systems become more integrated into daily life, the need for safer and more reliable moderation has never been greater. Large Language Models (LLMs) have demonstrated remarkable capabilities, surpassing earlier models in complexity and performance. Their evaluation across diverse tasks has consistently showcased their potential, enabling the development of adaptive and personalized agents. However, despite these advancements, LLMs remain prone to errors, particularly in areas requiring nuanced moral reasoning. They struggle with detecting implicit hate, offensive language, and gender biases due to the subjective and context-dependent nature of these issues. Moreover, their reliance on training data can inadvertently reinforce societal biases, leading to inconsistencies and ethical concerns in their outputs. To explore the limitations of LLMs in this role, we developed an experimental framework based on state-of-the-art (SOTA) models to assess human emotions and offensive behaviors. The framework introduces a unified benchmark dataset encompassing 49 distinct categories spanning the wide spectrum of human emotions, offensive and hateful text, and gender and racial biases. Furthermore, we introduced SafePhi, a QLoRA fine-tuned version of Phi-4, adapting diverse ethical contexts and outperforming benchmark moderators by achieving a Macro F1 score of 0.89, where OpenAI Moderator and Llama Guard score 0.77 and 0.74, respectively. This research also highlights the critical domains where LLM moderators consistently underperformed, pressing the need to incorporate more heterogeneous and representative data with human-in-the-loop, for better model robustness and explainability.
