Mitigating Errors in LLM-Generated Web API Invocations via Retrieval-Augmented Generation and Constrained Decoding

2026-07-07

AI Summary

This work addresses the challenge that large language models often generate erroneous Web API invocation code due to the complexity and dynamic evolution of API specifications. To mitigate this, the authors propose a novel approach combining retrieval-augmented generation (RAG) with constrained decoding (CD), wherein OpenAPI specifications are automatically compiled into regular expression constraints to guide code generation—a first in the field—and a compact, endpoint-focused retrieval mechanism tailored for OpenAPI is introduced. Experimental results demonstrate that constrained decoding substantially improves generation correctness by effectively preventing invalid URLs, HTTP methods, and parameters. While RAG reduces hallucination in full-call generation, it may introduce redundancy when the target endpoint is already known. This study establishes a new paradigm for specification-driven API code generation.

Abstract

Integration of web APIs is a cornerstone of modern software systems, yet writing correct web API invocation code remains challenging due to complex and evolving API specifications. Although LLMs are increasingly used for code generation, previous work has empirically shown that their ability to generate correct web API integrations is limited. At the same time, mitigation techniques and their effectiveness for this setting remain insufficiently understood. In this paper, we propose and systematically evaluate retrieval-augmented generation (RAG) and constrained decoding (CD) as two complementary approaches to improving LLM-generated web API invocation code. For RAG, we design a retriever that processes OpenAPI specifications and retrieves compact endpoint representations to inject into model prompts. For CD, we introduce an automatic translation from OpenAPI specifications to regex-based constraints enforced during generation. We evaluate both approaches on WAPIIBench's existing synthetic dataset and on a new real-world dataset derived from GitHub repositories. Our results show that RAG reduces hallucinations and improves correctness when generating full API invocations but reduces it when the endpoint is already provided as it encourages the generation of unnecessary parameters. In contrast, CD reliably prevents illegal URLs, HTTP methods, and arguments and substantially improves overall correctness for both starter codes.

Problem

• LLM-generated code
• Web API invocations
• error mitigation
• code correctness
• API integration

Innovation

• Retrieval-Augmented Generation
• Constrained Decoding
• OpenAPI Specification
• LLM Code Generation
• Web API Integration