Search papers, labs, and topics across Lattice.
The paper introduces DeepSight, an open-source toolkit integrating safety evaluation and diagnosis for large language models (LLMs) and multimodal LLMs (MLLMs). DeepSight comprises DeepSafe for safety evaluation and DeepScan for diagnosis, unifying task and data protocols to connect the two stages. The toolkit aims to provide white-box insights into model safety, addressing limitations of existing tools that treat evaluation and diagnosis as separate processes.
DeepSight offers an all-in-one open-source toolkit for LLM safety, promising to move beyond black-box evaluations and provide white-box insights into internal mechanisms.
As the development of Large Models (LMs) progresses rapidly, their safety is also a priority. In current Large Language Models (LLMs) and Multimodal Large Language Models (MLLMs) safety workflow, evaluation, diagnosis, and alignment are often handled by separate tools. Specifically, safety evaluation can only locate external behavioral risks but cannot figure out internal root causes. Meanwhile, safety diagnosis often drifts from concrete risk scenarios and remains at the explainable level. In this way, safety alignment lack dedicated explanations of changes in internal mechanisms, potentially degrading general capabilities. To systematically address these issues, we propose an open-source project, namely DeepSight, to practice a new safety evaluation-diagnosis integrated paradigm. DeepSight is low-cost, reproducible, efficient, and highly scalable large-scale model safety evaluation project consisting of a evaluation toolkit DeepSafe and a diagnosis toolkit DeepScan. By unifying task and data protocols, we build a connection between the two stages and transform safety evaluation from black-box to white-box insight. Besides, DeepSight is the first open source toolkit that support the frontier AI risk evaluation and joint safety evaluation and diagnosis.
