data analysis

This work addresses the significant limitations of spreadsheet-based analysis in reproducibility, auditability, version control, and automation. It proposes a migration pathway from Excel to research-grade analytical workflows by leveraging Python’s pandas library as a bridge. The study introduces an innovative set of Excel-to-pandas mapping rules, categorizes nine canonical workflow patterns, and compiles a catalog of common failure modes. Seven end-to-end real-world examples demonstrate the approach in practice. By retaining Excel as a familiar interface for input and output while integrating version control, automated refreshing, and seamless incorporation of statistical and machine learning methods, the proposed framework enables governed, reproducible, and auditable tabular data analysis.

Traditional boxplots struggle to detect bimodal or multimodal structures in univariate data and fail to reveal underlying subgroups or outliers. To address this, we propose the *bixplot*, a novel boxplot variant that integrates a univariate clustering algorithm with continuity constraints and a minimum cluster size requirement to automatically identify and visualize meaningful subgroups within multimodal distributions. The bixplot supports direct plotting of individual data points and enables gradient color mapping based on external variables, thereby enhancing interpretability of substructures. Crucially, it guarantees non-nested clusters and ensures each cluster contains a sufficient number of distinct observations. Implemented as open-source Python and R packages, the bixplot demonstrates superior performance over conventional boxplots across multiple real-world datasets—clearly exposing previously overlooked multimodal patterns and substantially improving the depth and practical utility of exploratory data analysis.

This work addresses the lack of standardized evaluation for large language models (LLMs) in generating visualization code from tabular data. We introduce PandasPlotBench, the first benchmark dataset specifically targeting Pandas DataFrame visualization—comprising 175 human-annotated tasks spanning Matplotlib, Seaborn, and Plotly. Our methodology employs natural-language instruction-driven code generation, coupled with standardized prompting and execution-based code evaluation. Key contributions include: (1) the first fine-grained, task-level evaluation framework for exploratory data visualization from tables; (2) empirical identification of a significant generalization gap for LLMs on Plotly—despite strong performance on Matplotlib and Seaborn; (3) demonstration of robustness to concise instructions across the latter two libraries; and (4) open-sourcing of both the benchmark dataset and evaluation framework, which has been widely adopted by the research community.

Traditional Tukey boxplots employ a fixed outlier detection rule that ignores sample size effects, and existing modifications lack a unified theoretical foundation. Method: We systematically reformulate the boxplot and its variants as visual implementations of multiple hypothesis testing, introducing a novel FDR-based framework—the first of its kind—to unify classical approaches and sample-size-adaptive strategies. This framework accommodates alternative multiple testing criteria (e.g., FWER, PFER) and seamlessly integrates robust location and scale estimators. Contribution/Results: The proposed framework substantially improves accuracy, consistency, and interpretability of outlier identification across diverse sample sizes. It bridges statistical rigor with practical usability, delivering a principled graphical tool for exploratory data analysis that maintains theoretical coherence while supporting flexible, robust implementation.

This study investigates common quality attributes characterizing high-impact Jupyter Notebooks. To address the lack of cross-platform empirical evidence on data science artifacts, we conduct the first comparative analysis of highly upvoted Kaggle notebooks and highly starred GitHub notebooks. Methodologically, we integrate exploratory data analysis, SHAP-based interpretability analysis, random forest feature importance assessment, and structural parsing of code and narrative text to systematically quantify three core dimensions: documentation completeness, modular design, and reproducibility configuration. Results show that these three features collectively account for over 75% of predictive importance for quality classification. Based on this, we propose an actionable Notebook quality assessment framework and a beginner-oriented practical guideline. This work establishes the first empirically grounded, cross-platform benchmark for evaluating Jupyter Notebooks—directly supporting pedagogy, peer review, and tool development in data science education and practice.

This study addresses the lack of systematic understanding regarding the effectiveness and usage practices of univariate distribution visualizations across diverse tasks and user groups. Through a mixed-methods approach—combining a click-based selection experiment and survey with 215 participants alongside in-depth interviews with five visualization practitioners—the work systematically evaluates the accuracy, user preferences, and common misinterpretations associated with boxplots, violin plots, jittered scatterplots, and histograms in typical analytical tasks. For the first time, it integrates task performance, subjective preference, and real-world practice, revealing a frequent mismatch between chart familiarity and task accuracy, thereby challenging the assumption that commonly used or conventional visualizations are inherently optimal. The findings demonstrate significant performance differences among chart types in low-level tasks, with widely adopted histograms and boxplots not consistently outperforming alternatives.

This work addresses the widespread misuse of statistical methods—often stemming from implicit or ambiguous assumptions—which exacerbates the reproducibility crisis in scientific research, particularly in hypothesis testing and meta-analysis where formal verification mechanisms are lacking. To bridge this gap, the authors propose the first formal verification framework tailored for Python-based statistical programs. By developing a Why3-py frontend, they translate dynamically typed, runtime-polymorphic Python code into the WhyML intermediate representation and extend the StatWhy tool to support meta-analysis verification. Integrating program transformation, static analysis, and formal verification techniques, this approach enables, for the first time, automated correctness verification of statistical programs written in Python, effectively uncovering overlooked assumptions and misuses, thereby filling a critical void in the formal verification of statistical software.

This study addresses a critical limitation in traditional reproducible research, where sharing only code and results fails to expose the implicit assumptions, expectations, and premises underlying an analyst’s reasoning—thereby hindering thorough evaluation of analytical quality. To overcome this, the paper proposes a formal modeling framework that explicitly translates the analyst’s tacit reasoning process into structured logical representations, statically capturing the construction logic of the analysis. This approach enables systematic scrutiny of the analytical chain of reasoning, assumption sensitivity, and conclusion robustness—even in the absence of the original data. Empirical validation on representative data analysis tasks demonstrates the framework’s effectiveness, achieving both logical visualization and data-free static assessment of analytical integrity.

This work addresses the challenge of reproducibility in data visualization scripts, which often lack essential components such as source code, input data, execution environment, or output artifacts. To bridge this gap, we propose yProv4DV—a lightweight Python library that, for the first time, targets script-based visualization workflows by automatically capturing comprehensive provenance information—including source code, input data, runtime environment, and output results—through a single function call. Designed to be minimally invasive and ready-to-use, yProv4DV enables full reproducibility of visualization outputs without requiring any modification to existing scripts. This approach significantly reduces the development burden on researchers striving to ensure reproducibility and fills a critical void in automated provenance support within visualization pipelines.

This study addresses the cognitive bias in scatterplots where “data-induced grouping”—arising from the interplay between data values and visual encoding—leads users to misinterpret spatial arrangements as meaningful patterns. Through two user studies, the authors systematically demonstrate the prevalence of this phenomenon, develop the first perceptual model capable of predicting whether users perceive a given set of points as a coherent group, and propose a visualization intervention strategy that integrates user perception with data reordering. Notably, the model effectively captures users’ tendency to group points based on trends even in nominal data contexts. Applied to visualization diagnosis and optimization, this approach significantly enhances the accuracy and reliability of graphical representations.