Search papers, labs, and topics across Lattice.
This study investigates the motivations behind algorithm mentions in natural language processing (NLP) academic papers, utilizing a sentence-level framework to classify and analyze these motivations. By employing manual annotation and machine learning techniques, the authors found that over half of algorithm-related sentences indicate direct usage, while motivations for improvement are the least common, with a notable shift over time from description to use. The findings highlight the evolving landscape of algorithm representation in research, revealing a decline in the diversity of motivations associated with individual algorithms.
More than half of algorithm mentions in NLP papers are for direct use, signaling a significant shift in how researchers engage with algorithms over time.
With the rise of data-intensive science, algorithms have become central to scientific research. In academic papers, algorithms are mentioned for different purposes, such as describing, using, comparing, or improving methods for specific research tasks. Identifying these purposes can reveal relationships among algorithms and help assess their roles and value. Taking natural language processing (NLP) as an example, this study proposes a sentence-level framework for identifying, analyzing, and tracing the evolution of motivations for mentioning algorithms. We first identify algorithm entities and algorithm-related sentences from full-text papers through manual annotation and machine learning. We then classify mention motivations using pretrained models and data augmentation, and analyze their distribution and temporal evolution. The results show that deep learning models trained with augmented data outperform traditional machine learning models in motivation classification. In NLP papers, more than half of algorithm-related sentences express direct use, whereas improvement is the least frequent motivation. The diversity of motivations has increased over time. For specific algorithm categories, grammar-based algorithms are more often mentioned for description, while machine learning algorithms are more often mentioned for use. Over time, use motivations have gradually replaced description motivations across different algorithms, and the number of motivation types associated with individual algorithms has declined significantly. This study reveals how authors mention algorithm entities in academic writing and provides a basis for future research on algorithm relationship identification and algorithm impact evaluation.
