Search papers, labs, and topics across Lattice.
SciZoom, a new benchmark for hierarchical scientific summarization, is introduced to address the limitations of existing datasets in the LLM era. The benchmark comprises 44,946 papers from top ML venues, stratified into pre- and post-LLM eras, and provides three levels of summarization targets: Abstract, Contributions, and TL;DR. Linguistic analysis of the dataset reveals significant shifts in scientific writing patterns after the advent of LLMs, including increased use of formulaic expressions and a decline in hedging.
LLM-assisted scientific writing is producing more confident but homogenized prose, as evidenced by a 23% decline in hedging in the post-LLM era.
The explosive growth of AI research has created unprecedented information overload, increasing the demand for scientific summarization at multiple levels of granularity beyond traditional abstracts. While LLMs are increasingly adopted for summarization, existing benchmarks remain limited in scale, target only a single granularity, and predate the LLM era. Moreover, since the release of ChatGPT in November 2022, researchers have rapidly adopted LLMs for drafting manuscripts themselves, fundamentally transforming scientific writing, yet no resource exists to analyze how this writing has evolved. To bridge these gaps, we introduce SciZoom, a benchmark comprising 44,946 papers from four top-tier ML venues (NeurIPS, ICLR, ICML, EMNLP) spanning 2020 to 2025, explicitly stratified into Pre-LLM and Post-LLM eras. SciZoom provides three hierarchical summarization targets (Abstract, Contributions, and TL;DR) achieving compression ratios up to 600:1, enabling both multi-granularity summarization research and temporal mining of scientific writing patterns. Our linguistic analysis reveals striking shifts in phrase patterns (up to 10x for formulaic expressions) and rhetorical style (23% decline in hedging), suggesting that LLM-assisted writing produces more confident yet homogenized prose. SciZoom serves as both a challenging benchmark and a unique resource for mining the evolution of scientific discourse in the generative AI era. Our code and dataset are publicly available on GitHub (https://github.com/janghana/SciZoom) and Hugging Face (https://huggingface.co/datasets/hanjang/SciZoom), respectively.
