Improving LLM-Driven Test Generation by Learning from Mocking Information

2026-04-21

AI Summary

This work addresses a critical limitation in existing large language model (LLM)-based unit test generation approaches, which often overlook the mocking information embedded in developer-written tests. To bridge this gap, we propose MOCKMILL, the first method that leverages mocking details as guiding signals to automatically extract and utilize such information for directing LLMs to generate high-quality test cases targeting mocked components. MOCKMILL integrates an iterative generation-and-repair mechanism to ensure the executability of the produced tests. Empirical evaluation on ten classes across six Java projects demonstrates that MOCKMILL substantially improves code coverage, effectively supplements code lines missed by both baseline techniques and existing tests, and kills a significantly higher number of mutants.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Large Language Models (LLMs) have recently shown strong potential for automated unit test generation. This has motivated us to investigate whether developer-defined test doubles (commonly referred to as mocks) available in existing test suites can be leveraged to improve LLM-driven test generation. To this end, we propose MOCKMILL, an LLM-based technique and tool that generates test cases by exploiting mocking information automatically extracted from developer-written tests. MOCKMILL targets components that are replaced by test doubles in existing tests and uses the encoded stubbings and interaction expectations to guide test generation, combined with an iterative generation-and-repair process to ensure executable tests. We evaluated MOCKMILL on 10 open-source classes from six Java projects using four LLMs, and compared the generated tests with existing project tests and tests produced by baseline approaches. The results show that MOCKMILL's tests cover lines of code and kill mutants that existing tests and baseline-generated tests miss. Overall, our findings provide preliminary evidence that leveraging mocking information is a complementary and effective way to enhance LLM-based test generation.

Problem

• LLM-driven test generation
• mocking information
• test doubles
• unit test generation
• automated testing

Innovation

• LLM-based test generation
• mocking information
• test doubles
• iterative generation-and-repair
• automated unit testing