Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper formalizes the conflicts and dilemmas faced by LLMs as a priority graph, where nodes represent instructions/values and edges represent context-specific priorities. The analysis reveals that achieving a unified and stable LLM alignment is challenging due to the dynamic and inconsistent nature of the graph across different contexts. The authors further identify a "priority hacking" vulnerability where adversarial contexts can manipulate the graph, and propose a runtime verification mechanism to mitigate this.
LLM alignment is fundamentally challenged by the dynamic and inconsistent nature of their internal "priority graphs," which adversaries can exploit through context manipulation.
As Large Language Models (LLMs) become more powerful and autonomous, they increasingly face conflicts and dilemmas in many scenarios. We first summarize and taxonomize these diverse conflicts. Then, we model the LLM's preferences to make different choices as a priority graph, where instructions and values are nodes, and the edges represent context-specific priorities determined by the model's output distribution. This graph reveals that a unified stable LLM alignment is very challenging, because the graph is neither static nor necessarily consistent in different contexts. Besides, it also reveals a potential vulnerability: priority hacking, where adversaries can craft deceptive contexts to manipulate the graph and bypass safety alignments. To counter this, we propose a runtime verification mechanism, enabling LLMs to query external sources to ground their context and resist manipulation. While this approach enhances robustness, we also acknowledge that many ethical and value dilemmas are philosophically irreducible, posing a long-term, open challenge for the future of AI alignment.