Testing Deep Learning Library APIs via Cross-Framework Differential Fuzzing

2026-08-12

AI Summary

This work addresses the challenge that existing testing approaches for deep learning library APIs struggle to uncover functional bugs that manifest consistently across hardware but exhibit behavioral discrepancies across frameworks. The authors propose Xamt, the first systematic effort to construct and validate 676 groups of functionally equivalent APIs—spanning 2,563 APIs across seven mainstream deep learning libraries—and detect inconsistencies via cross-framework differential fuzzing. Key innovations include establishing cross-framework API equivalence, normalizing parameter roles, group-level execution validation, and a variance-guided strategy for generating diverse input types. In experiments, Xamt uncovered 72 reproducible issues, including four crashes and 68 output inconsistencies; 25 have been confirmed by developers, and 23 have already been fixed.

Abstract

Deep learning libraries underpin many safety- and reliability-critical applications, yet existing API-level testing techniques often rely on intra-library properties or CPU--GPU differential oracles and may miss defects that behave consistently across hardware backends. We present Xamt, a cross-framework differential fuzzing approach for deep learning library APIs. Xamt constructs and tests execution-validated groups of APIs intended to implement equivalent operations across seven libraries. It uses explicit API aliases and parameter-role normalization to construct candidate correspondences and validates them through pairwise execution and a group-level behavioral check on canonical ordinary inputs. The resulting groups are explored using variance-guided differential fuzzing with ordinary, boundary, and non-finite inputs. Crash and inconsistency oracles flag executions exhibiting abnormal termination or inconsistent outputs for subsequent reproduction and analysis. Across the seven libraries, Xamt constructs 676 execution-validated groups containing 2,563 matched APIs. Among these, Xamt identifies 72 independently reproduced discrepancy cases, including 4 crash cases and 68 output inconsistencies. Among the 72 developer reports, 25 have been confirmed, including 23 that have been fixed.

Problem

• deep learning libraries
• API testing
• cross-framework differential
• output inconsistency
• fuzzing

Innovation

• cross-framework differential fuzzing
• API equivalence validation
• variance-guided fuzzing
• deep learning library testing
• execution-validated API groups