Investigating Metamorphic Fuzz Oracle Enhancement via Large Language Models

2026-06-12

AI Summary

This work addresses a key limitation in existing greybox fuzzing, which relies solely on crash signals as oracles and overlooks the semantic behavior of library functions, thereby constraining its defect detection capability. To overcome this, the authors propose MetaFOE, a novel framework that integrates large language models with metamorphic testing to automatically construct and embed semantics-aware oracles based on metamorphic relations. Leveraging diverse prompting strategies, MetaFOE generates 3,475 metamorphic relations—77.3% of which are validated as effective—and derives 6,228 metamorphic drivers. Evaluated on standard benchmarks, MetaFOE achieves an average edge coverage improvement of 18.7% and uncovers 1,528 unique crashes, substantially outperforming conventional fuzzing approaches.

Abstract

Fuzz drivers are essential components of greybox fuzzing, as they encapsulate target interfaces, define test spaces, and largely determine fuzzing effectiveness. Existing fuzz drivers typically rely on crash-based oracles for security testing, overlooking library functionality and limiting bug detection capability. In this paper, we present the first study on metamorphic-based fuzz oracle enhancement (MFOE), which augments existing fuzz drivers with metamorphic-based oracles derived from metamorphic relations (MRs). Since constructing and integrating such oracles requires substantial domain knowledge, automating MFOE is challenging. To address this challenge, we propose MetaFOE, an LLM-based framework that automatically generates and integrates metamorphic-based oracles. We evaluate MetaFOE on OSS-Fuzz drivers using three modern LLMs and five prompt strategies. MetaFOE generates 3,475 MRs, of which 77.3% are applicable, and implements 12,351 meta drivers, with 6,228 being valid. After three hours of fuzzing, the valid meta drivers improve edge coverage by an average of 18.7% and trigger 1,528 unique crashes. Our results demonstrate both the effectiveness of metamorphic-based oracle enhancement and the feasibility of using LLMs to automate MFOE, providing valuable insights for advancing greybox fuzzing.

Problem

• metamorphic fuzz oracle
• greybox fuzzing
• fuzz driver
• bug detection
• oracle enhancement

Innovation

• metamorphic testing
• fuzz oracle
• large language models
• greybox fuzzing
• automated program analysis