Search papers, labs, and topics across Lattice.
This study investigates the differences in problem-solving processes between expert and novice data scientists by analyzing 440 Jupyter notebooks using multi-level sequence analysis. The research maps low-level coding actions to high-level problem-solving practices, revealing that experts and novices transition between data science phases similarly. However, experts exhibit shorter, more iterative workflows with efficient, context-specific action sequences, contrasting with novices' longer, linear processes.
Experts don't just code better, they iterate smarter: data science masters weave shorter, more agile workflows compared to novices' linear slogs.
The development of data science expertise requires tacit, process-oriented skills that are difficult to teach directly. This study addresses the resulting challenge of empirically understanding how the problem-solving processes of experts and novices differ. We apply a multi-level sequence analysis to 440 Jupyter notebooks from a public dataset, mapping low-level coding actions to higher-level problem-solving practices. Our findings reveal that experts do not follow fundamentally different transitions between data science phases than novices (e.g., Data Import, EDA, Model Training, Visualization). Instead, expertise is distinguished by the overall workflow structure from a problem-solving perspective and cell-level, fine-grained action patterns. Novices tend to follow long, linear processes, whereas experts employ shorter, more iterative strategies enacted through efficient, context-specific action sequences. These results provide data science educators with empirical insights for curriculum design and assessment, shifting the focus from final products toward the development of the flexible, iterative thinking that defines expertise-a priority in a field increasingly shaped by AI tools.
