Search papers, labs, and topics across Lattice.
This paper introduces the concept of prompting complexity, which quantifies the shortest plausible prompts required for deterministic decoding in instruction-tuned language models. By framing prompts as programs and leveraging the model's weights and training distribution, the authors establish a non-universal measure that varies across different models. Key findings include the computability of this complexity in finite contexts and the introduction of behavioral prompting complexity, paving the way for a structured research agenda on prompt optimization and accessibility of outputs.
Short prompts can unlock complex outputs in LLMs, but their efficiency varies dramatically across different models, challenging our understanding of prompt engineering.
In this paper, we define the quantity of prompting complexity: for a fixed instruction-tuned language model, what is the shortest plausible prompt that makes deterministic decoding produce a target text? It is an LM-relative analogue of resource-bounded Kolmogorov complexity: the prompt is a program, the model interface is the interpreter, and information omitted from the prompt is supplied by the model's weights, training distribution, tokenizer, template, and decoding rule. Unlike classical Kolmogorov complexity, this measure is intentionally non-universal. In the finite-context setting it is computable by enumeration, but there is no model-independent invariance theorem; the same text may be cheap for one model and inaccessible or expensive for another. To keep the search space aligned with prompt engineering, we restrict programs to plausible human-readable texts rather than arbitrary token strings. We extend the exact definition to soft prompting complexity for approximate outputs, yielding a lossy notion of model-relative text compression and a formal target for prompt optimization. We also define prompting distance by comparing shortest generating prompts, and behavioral prompting complexity for reaching any output satisfying a specification. Based on these formulations, we define a research agenda for empirically studying which texts and behaviors are accessible from short plausible prompts under a fixed LM interface.
