Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study investigates self-modeling in large language models (LLMs), specifically their ability to accurately answer questions about their own behavior, such as the impact of prompt edits on responses. A new benchmark was introduced to evaluate various self-modeling capabilities, revealing that existing models exhibit limited skill and make systematic errors on straightforward counterfactual queries. To enhance self-modeling abilities, the authors developed a synthetic data generation pipeline and employed reinforcement learning, achieving improvements across multiple model families, though these gains do not imply true introspection of the models' internal processes.
LLMs struggle with self-awareness, making systematic errors on simple counterfactuals, but reinforcement learning can boost their self-modeling skills.
We study self-modeling: an LLM's ability to answer questions about its own behavior. We focus on verifiable behavioral questions, such as whether a prompt edit would change the model's final answer. To measure this capability, we introduce a benchmark that tests diverse types of self-modeling questions. Current models show non-trivial but limited self-modeling skill, and make systematic mistakes on simple counterfactual questions about their own behavior. To improve self-modeling skill, we develop a scalable synthetic-data pipeline that produces self-modeling training data, and show that reinforcement-learning can improve aggregate self-modeling skill across three open-source model families with some transfer to held-out tasks. These gains, however, do not seem to constitute introspection consistently: improved self-modeling may not arise from privileged access to the model's internal decision process.