Search papers, labs, and topics across Lattice.
This paper introduces a computational model that predicts human visual attention on words in source code, leveraging a novel loss function to align neural network attention with eye-tracking data from software engineers. The model was evaluated across three datasets, achieving substantial improvements in performance over baseline methods, with increases in Pearson correlation of 64%, 16%, and an impressive 467%. Additionally, it demonstrates superior capabilities in scanpath prediction and outperforms existing models like Claude and GPT-5 in both reading and writing tasks, highlighting its effectiveness in understanding programmer cognitive processes.
Eye-tracking data reveals that our model predicts programmer attention with unprecedented accuracy, outperforming existing AI models by significant margins.
This paper presents a computational model to predict human visual attention over words in software source code. The visual attention of software engineers when reading source code has long been studied as a means to understand human cognitive processes during software engineering tasks. Predicting this visual attention is important for perfecting user interface design and understanding what information human programmers need. We propose a model of programmer visual attention in which we design a novel loss function that computes similarity between human attention measured during eye tracking experiments and the internal attention of the artificial neural network. We evaluate our model by comparing its outputs to actual eye tracking data from three separate datasets. Two are in the Java programming language and one is in the C programming language. Our model outperforms the baseline in software engineering by 64%, 16%, and 467% in each of these studies according to Pearson correlation. We used scanpath prediction as an example to demonstrate that our model is more capable of the task that requires the understanding of human thought process. Our model achieves a statistically significant improvement over the close baseline in the reading task according to normalized Levenshtein distance and outperforms both Claude and GPT-5 on both reading and writing tasks.
