Search papers, labs, and topics across Lattice.
This study investigates the impact of Large Language Models (LLMs) on the complexity metrics of scientific texts by analyzing millions of arXiv abstracts from 2010 to 2025, with a focus on the year 2023 as a pivotal moment for LLM adoption. The findings reveal a significant increase in a composite LLM-associated style index post-2023, alongside subtle shifts in the exponents of Zipf's and Heaps' laws, indicating nuanced changes in linguistic dynamics. Notably, the research highlights a marked increase in turnover among top-ranked words and a flattening relationship between the LLM style index and complexity metrics, suggesting that LLMs are reshaping the emergent properties of scientific language in a human-AI collaborative context.
LLMs are reshaping the complexity of scientific language, evidenced by increased word turnover and altered relationships between style and complexity metrics.
While human language has long been studied as a complex system, Large Language Models (LLMs) are rapidly becoming contributors to its dynamics. Because LLMs are trained on human language use, their effects on the broader human-AI linguistic ecosystem are likely subtle at first. As their use becomes more widespread, however, LLMs may alter emergent properties of language, particularly as models increasingly train on mixed human-LLM textual data. Here, we draw on complexity science to look for subtle LLM effects in millions of arXiv abstracts from 2010 to 2025. The year 2023, when LLMs rapidly became widely used, serves as a landmark in a natural experiment. While we find a sharp increase in a composite LLM-associated style index after early 2023, we observe only subtle changes in the exponents of Zipf's law and Heaps' law. More compelling, however, are two subtle changes in complexity metrics that emerge from 2023 onward. First, turnover among top-ranked words increases sharply. Second, the positive relationship between the LLM-associated style index and three complexity metrics--vocabulary size and the exponents of Heaps' and Zipf's laws--becomes flatter after 2022. Together, these patterns are consistent with changes in the emergent properties of scientific text in a mixed human-AI linguistic ecosystem.
