Search papers, labs, and topics across Lattice.
This paper investigates the ability of GPT-5 to perform interpretative citation context analysis (CCA) by focusing on a single challenging case and varying prompt scaffolding. A two-stage pipeline was implemented, involving a citation-text-only classification followed by cross-document interpretative reconstruction using full texts. The study found that while GPT-5 can generate a space of plausible interpretations, prompt engineering significantly influences the model's attention, vocabulary, and ultimately, the types of readings it favors, highlighting the importance of prompt-sensitivity analysis.
Prompt engineering can systematically bias LLMs' interpretations of citation contexts, tilting them toward strained readings and specific vocabularies.
This paper tests whether large language models (LLMs) can support interpretative citation context analysis (CCA) by scaling in thick, text-grounded readings of a single hard case rather than scaling up typological labels. It foregrounds prompt-sensitivity analysis as a methodological issue by varying prompt scaffolding and framing in a balanced 2x3 design. Using footnote 6 in Chubin and Moitra (1975) and Gilbert's (1977) reconstruction as a probe, I implement a two-stage GPT-5 pipeline: a citation-text-only surface classification and expectation pass, followed by cross-document interpretative reconstruction using the citing and cited full texts. Across 90 reconstructions, the model produces 450 distinct hypotheses. Close reading and inductive coding identify 21 recurring interpretative moves, and linear probability models estimate how prompt choices shift their frequencies and lexical repertoire. GPT-5's surface pass is highly stable, consistently classifying the citation as "supplementary". In reconstruction, the model generates a structured space of plausible alternatives, but scaffolding and examples redistribute attention and vocabulary, sometimes toward strained readings. Relative to Gilbert, GPT-5 detects the same textual hinges yet more often resolves them as lineage and positioning than as admonishment. The study outlines opportunities and risks of using LLMs as guided co-analysts for inspectable, contestable interpretative CCA, and it shows that prompt scaffolding and framing systematically tilt which plausible readings and vocabularies the model foregrounds.
