JupOtter: Cell-Level Bug Detection in Jupyter Notebooks

2026-06-22

AI Summary

This work addresses the challenge that existing static analysis tools and large language models struggle to effectively detect cell-level code defects in Jupyter Notebooks. To overcome this limitation, the authors propose a fine-grained defect detection system that employs a structure-aware tokenization strategy to preserve notebook cell boundaries and integrates a machine learning–driven cell-level classification model for accurate prediction. The key contributions include a notebook-specific structure-preserving tokenization method, a dedicated cell-level defect prediction mechanism, and the construction of OtterDataset, a new benchmark comprising over 21,000 annotated samples. Experimental results demonstrate that the proposed approach achieves significantly higher F1 scores than both conventional static analyzers and large language models on two out of three benchmark datasets.

Abstract

Jupyter Notebooks are an increasingly popular coding environment used across many domains, especially in Python-based data science and scientific computing. Originally used for prototyping and interactive exploration, notebooks are increasingly used to develop more complex programs, leading to a rapid rise in buggy notebooks on platforms like GitHub. To address this trend, we present JupOtter, a bug detection system designed specifically for Jupyter Notebooks. JupOtter features three novel contributions: (1) a notebook-specific tokenization strategy that preserves cell structure, (2) a cell-level bug prediction technique, and (3) a new labeled dataset, OtterDataset, containing over 21,000 notebooks annotated for fine-grained cell-level bug detection. JupOtter achieves cell-level bug detection F1 scores that surpass static analyzers and large language models in two out of three evaluation datasets.

Problem

• Jupyter Notebooks
• bug detection
• cell-level
• software bugs
• notebook analysis

Innovation

• cell-level bug detection
• Jupyter Notebooks
• notebook tokenization
• OtterDataset
• static analysis