Harnessing LLMs for Document-Guided Fuzzing of Python Libraries
This work addresses the challenge of testing Python library APIs, which often involve complex parameter constraints and cross-parameter dependencies that existing methods struggle to model effectively. The authors propose VistaFuzz, a novel approach that leverages locally deployed open-source large language models to automatically extract parameter specifications and dependency relationships directly from API documentation. These extracted constraints are then integrated into the fuzzing input generation process, enabling documentation-guided, high-efficiency testing. By combining natural language parsing, constraint reasoning, and fuzz testing, VistaFuzz achieves over 95% effective test case generation across 7,718 APIs from 12 widely used Python libraries. The evaluation uncovered 74 distinct issues, 43 of which were confirmed by developers and 29 subsequently fixed.