Search papers, labs, and topics across Lattice.
This paper investigates harm amplification in large language model (LLM) interactions, highlighting how these models can enable malicious users to produce harmful content through multi-turn conversations. The authors introduce HarmAmp, a benchmark designed to assess harm amplification across twelve risk categories, and propose TrajSafe, a proactive monitoring system that mitigates harmful trajectories while maintaining model performance. Experimental results show that TrajSafe effectively reduces harmful outcomes in multi-turn interactions without significantly increasing the rate of refusals or compromising the model's general capabilities.
HarmAmp reveals that LLMs can amplify harm in multi-turn interactions, while TrajSafe effectively curbs this risk without sacrificing performance.
Large language models (LLMs) can serve as helpful assistants, yet they can equally function as harm amplifiers that enable malicious users to achieve harmful outcomes beyond their capabilities through extended interactions. This risk manifests along two axes, i.e., democratizing domain expertise that allows novices to produce specialized harmful content, and scaling harmful operations at volumes that manual effort cannot match. Existing works, however, often overlook how LLMs compound harm across multi-turn conversations. We introduce HarmAmp, a new benchmark for multi-turn harm amplification scenarios spanning twelve risk categories. Each scenario is grounded in real-world threats and satisfies rigorous criteria, i.e., substantive amplification, operational specificity, and multi-turn necessity. We further propose TrajSafe, a proactive monitor that anticipates harmful trajectories and intervenes through actions such as probing users' genuine intents and steering the models towards safer completion. Our extensive experiments demonstrate that TrajSafe significantly reduces the harmfulness incurred in multi-turn interactions while preserving a low over-refusal rate and the target model's general capabilities. Our work offers a promising paradigm to alleviate the nuanced safety risks in LLM interactions.
