Search papers, labs, and topics across Lattice.
JupOtter is a specialized bug detection system for Jupyter Notebooks that introduces a unique tokenization strategy tailored to preserve cell structure, enabling effective cell-level bug prediction. By leveraging a newly created dataset, OtterDataset, which includes over 21,000 annotated notebooks, JupOtter outperforms traditional static analyzers and large language models in detecting bugs. This advancement is crucial as it addresses the growing complexity of notebooks in data science, providing a targeted solution for improving code reliability.
JupOtter achieves superior bug detection in Jupyter Notebooks, outperforming existing tools by focusing on cell-level analysis.
Jupyter Notebooks are an increasingly popular coding environment used across many domains, especially in Python-based data science and scientific computing. Originally used for prototyping and interactive exploration, notebooks are increasingly used to develop more complex programs, leading to a rapid rise in buggy notebooks on platforms like GitHub. To address this trend, we present JupOtter, a bug detection system designed specifically for Jupyter Notebooks. JupOtter features three novel contributions: (1) a notebook-specific tokenization strategy that preserves cell structure, (2) a cell-level bug prediction technique, and (3) a new labeled dataset, OtterDataset, containing over 21,000 notebooks annotated for fine-grained cell-level bug detection. JupOtter achieves cell-level bug detection F1 scores that surpass static analyzers and large language models in two out of three evaluation datasets.
