Search papers, labs, and topics across Lattice.
The paper introduces the "conceptual multiverse," an interactive system designed to expose and allow manipulation of the hidden conceptual decisions language models make when answering open-ended questions. This system visualizes decision spaces related to framing questions and values, enabling users to inspect, intervene, and verify these decisions against domain reasoning. The authors demonstrate the utility of the conceptual multiverse across philosophy, AI alignment, and poetry, showing that it helps users develop a more comprehensive understanding of the problem space.
Uncover the hidden assumptions baked into LLM responses with a new interactive system that lets you explore alternative conceptual framings and values.
When language models answer open-ended problems, they implicitly make hidden decisions that shape their outputs, leaving users with uncontextualized answers rather than a working map of the problem; drawing on multiverse analysis from statistics, we build and evaluate the conceptual multiverse, an interactive system that represents conceptual decisions such as how to frame a question or what to value as a space users can transparently inspect, intervenably change, and check against principled domain reasoning; for this structure to be worth navigating rather than misleading, it must be rigorous and checkable against domain reasoning norms, so we develop a general verification framework that enforces properties of good decision structures like unambiguity and completeness calibrated by expert-level reasoning; across three domains, the conceptual multiverse helped participants develop a working map of the problem, with philosophy students rewriting essays with sharper framings and reversed theses, alignment annotators moving from surface preferences to reasoning about user intent and harm, and poets identifying compositional patterns that clarified their taste.
MIT CSAIL