Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces DISEF, a Dual-Stage Instruction Safety Evaluation Framework, to assess the vulnerability of LLMs to jailbreaking attacks in Chinese-language settings. DISEF uses Virtualized Scenario Embedding (VSE) to test alignment stability under contextual shifts and Formal Payload Splitting (FPS) to analyze robustness against fragmented or implicitly encoded risk-related content. Experiments on the IJCAI 2025 benchmark reveal vulnerabilities in multiple LLMs, providing insights for improving safety alignment and threat detection.
LLMs exhibit surprising vulnerabilities to jailbreaking attacks in Chinese, even when prompts are embedded in seemingly benign contexts or when risky content is fragmented, revealing critical gaps in current safety alignment strategies.
Large Language Models (LLMs) equipped with advanced reasoning capabilities have demonstrated impressive performance across natural language tasks, yet remain susceptible to context-dependent or partially obfuscated safety-sensitive instructions, particularly in Chinese-language settings. To systematically assess these risks, this paper introduces a Dual-Stage Instruction Safety Evaluation Framework (DISEF) comprising Virtualized Scenario Embedding (VSE), which embeds queries into semantically benign contexts to examine alignment stability under scenario-driven shifts, and Formal Payload Splitting (FPS), a controlled diagnostic technique for analyzing robustness when models process fragmented or implicitly encoded risk-related content. The framework is validated using the IJCAI 2025 Generative LLM Security Attack-Defense benchmark, covering prompt diversity, risk-consistency assessment, and content-level risk distribution across multiple representative LLMs. Experimental findings reveal notable discrepancies in alignment robustness, highlighting cross-model vulnerability patterns and exposure points within Chinese instruction-processing pathways. The proposed framework provides actionable insights for strengthening safety alignment, enhancing threat detection mechanisms, and supporting the development of standardized evaluation approaches for next-generation generative AI systems.