One Documentation Does Not Fit All: Case Study of TensorFlow Documentation

2025-05-03

AI Summary

TensorFlow documentation exhibits low usability for machine learning (ML) practitioners without software engineering backgrounds. Method: We propose the first dual-dimensional “Nature & Trigger” taxonomy for ML software documentation—categorizing questions by *nature* (e.g., conceptual, debugging, usage) and *trigger* (e.g., example insufficiency, error occurrence)—and conduct an empirical study integrating qualitative content analysis, Stack Overflow data mining, and thematic coding. Results: 64.3% of documentation-related questions stem from inadequate or non-transferable code examples; 24.9% concern error diagnosis and debugging. Customized tutorials fail to improve effectiveness, exposing a fundamental tension between *generality* and *user adaptability*. The study systematically identifies structural deficiencies in current ML documentation—particularly in error support and generalizability—and establishes a transferable documentation assessment framework. This work provides both theoretical foundations and actionable design guidelines for improving ML tool documentation.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• User Modeling, Personalization and Recommendation: ML for personalized search and recommendations
• Web Mining and Content Analysis: Machine learning and data science for the Web
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Software documentation guides the proper use of tools or services. With the rapid growth of machine learning libraries, individuals from various fields are incorporating machine learning into their workflows through programming. However, many of these users lack software engineering experience, affecting the usability of the documentation. Traditionally, software developers have created documentation primarily for their peers, making it challenging for others to interpret and effectively use these resources. Moreover, no study has specifically focused on machine learning software documentation or analyzing the backgrounds of developers who rely on such documentation, highlighting a critical gap in understanding how to make these resources more accessible. This study examined customization trends in TensorFlow tutorials and compared these artifacts to analyze content and design differences. We also analyzed Stack Overflow questions related to TensorFlow documentation to understand the types of questions and the backgrounds of the developers asking them. Further, we developed two taxonomies based on the nature and triggers of the questions for machine learning software. Our findings showed no significant differences in the content or the nature of the questions across different tutorials. Our results show that 24.9% of the questions concern errors and exceptions, while 64.3% relate to inadequate and non-generalizable examples in the documentation. Despite efforts to create customized documentation, our analysis indicates that current TensorFlow documentation does not effectively support its target users.

Problem

• Examining usability gaps in TensorFlow documentation for diverse users
• Analyzing developer backgrounds and question trends on Stack Overflow
• Assessing effectiveness of customized tutorials for machine learning tools

Innovation

• Analyzed TensorFlow documentation customization trends
• Developed taxonomies for machine learning software questions
• Evaluated Stack Overflow questions on TensorFlow documentation