Search papers, labs, and topics across Lattice.
This paper presents a large-scale analysis of LLM reasoning traces in binary vulnerability analysis, revealing structured, token-level patterns that govern exploration. By analyzing 99,563 reasoning steps across 521 binaries, the authors identify four dominant implicit patterns: early pruning, path-dependent lock-in, targeted backtracking, and knowledge-guided prioritization. These patterns demonstrate that LLM-driven binary analysis is organized through implicit decisions rather than explicit control flow, offering a new abstraction for understanding LLM reasoning.
LLMs analyzing binaries aren't just spitting out tokens – they're exhibiting surprisingly structured reasoning patterns like "early pruning" and "targeted backtracking" that could revolutionize how we understand and control these systems.
Binary vulnerability analysis is increasingly performed by LLM-based agents in an iterative, multi-pass manner, with the model as the core decision-maker. However, how such systems organize exploration over hundreds of reasoning steps remains poorly understood, due to limited context windows and implicit token-level behaviors. We present the first large-scale, trace-level study showing that multi-pass LLM reasoning gives rise to structured, token-level implicit patterns. Analyzing 521 binaries with 99,563 reasoning steps, we identify four dominant patterns: early pruning, path-dependent lock-in, targeted backtracking, and knowledge-guided prioritization that emerge implicitly from reasoning traces. These token-level implicit patterns serve as an abstraction of LLM reasoning: instead of explicit control-flow or predefined heuristics, exploration is organized through implicit decisions regulating path selection, commitment, and revision. Our analysis shows these patterns form a stable, structured system with distinct temporal roles and measurable characteristics. Our results provide the first systematic characterization of LLM-driven binary analysis and a foundation for more reliable analysis systems.
