openai api

This study systematically investigates, for the first time, core practical challenges faced by developers using the OpenAI API—namely, prompt engineering complexity, token-level cost unpredictability, output non-determinism, and model opacity—previously lacking empirical grounding. Method: Leveraging 2,874 high-quality Stack Overflow Q&A pairs, we apply human annotation, LDA topic modeling, statistical analysis, and qualitative induction to construct a nine-category problem taxonomy and identify fine-grained challenges within each. Contribution/Results: We propose the first empirically grounded framework characterizing LLM usage difficulties for API practitioners, revealing a fundamental triadic tension among cost efficiency, controllability, and explainability. Our findings yield actionable optimization strategies for developers, concrete recommendations for API vendors to improve design and documentation, and novel research directions—including human-AI collaboration and trustworthy LLMs—for the broader community.

This work addresses the inefficiency, error-proneness, and technology-stack dependency developers face when authoring and maintaining OpenAPI specifications (OAS). To overcome these challenges, we propose OOPS—the first language- and framework-agnostic, LLM-driven static analysis approach. OOPS constructs an API dependency graph to link cross-file code artifacts and employs a two-stage pipeline—endpoint extraction followed by OAS generation—augmented with a self-refinement mechanism to mitigate LLM context limitations and reduce hallucinations. Evaluated on 12 real-world REST APIs spanning five programming languages and eight frameworks, OOPS achieves F1 scores of 98%, 97%, and 92% in endpoint identification, request/response structure inference, and parameter constraint derivation, respectively, with average input and output token counts below 5.6K and 0.9K, substantially decreasing reliance on manual intervention and handcrafted rules.

Online API documentation is predominantly published in unstructured HTML, necessitating substantial manual effort to convert it into structured specifications (e.g., OpenAPI) for AI agents and automated systems. Method: We propose a collaborative conversion framework integrating domain knowledge, large language models (LLMs), and web-structure-aware rules—enabling the first end-to-end, high-accuracy OpenAPI generation from lengthy, heterogeneous API documentation. Our approach employs multi-stage prompt engineering, structure-guided rule constraints, and rigorous format validation to significantly improve LLM output accuracy and consistency. Contribution/Results: Evaluated on hundreds of real-world API documentation pages, our method achieves full semantic coverage of original specifications. Deployed enterprise-wide, it has saved over thousands of person-hours of manual effort. The generated OpenAPI specifications robustly support LLM-driven service integration and tool-mediated API invocation.

To address challenges in dynamic system integration—including difficulty in API endpoint discovery and high token overhead in retrieval-augmented generation (RAG)—this paper proposes a summary-based Discovery Agent architecture. First, OpenAPI specifications are processed via format-aware, LLM-driven intelligent chunking to generate semantically compact endpoint summaries. Second, an efficient retrieval pipeline combines summary-based pre-filtering with on-demand loading of fine-grained details. This work presents the first systematic evaluation of OpenAPI chunking strategies for RAG-based endpoint discovery. On the RestBench benchmark, our method significantly improves F1 score (+12.3%) and precision (+15.6%) over naive chunking, maintains high recall (92.4%), and reduces token consumption by 68%. Key contributions include: (1) a format-aware chunking paradigm for OpenAPI documents; (2) an endpoint-level summary pre-filtering mechanism; and (3) a lightweight, efficient RAG framework tailored for API endpoint discovery.

REST API documentation frequently suffers from incompleteness, obsolescence, or inaccessibility, hindering both automated testing efficiency and human comprehension. This paper introduces the first LLM-driven, end-to-end framework for OpenAPI specification inference and black-box API testing—requiring only an API name and an LLM API key. It automatically generates and mutates HTTP requests, then infers specifications and detects defects via response analysis. A novel context-aware prompt masking strategy enables zero-shot discovery of undocumented routes and parameters without model fine-tuning. Evaluated on a standardized benchmark, the framework achieves 85.05% average recall for GET routes and 81.05% for query parameters, successfully uncovering hidden endpoints and diverse server-side errors (e.g., 5xx, logic flaws). The inferred OpenAPI specifications are directly compatible with mainstream API testing tools, enabling seamless integration into existing CI/CD and security validation pipelines.

This work addresses the limitations of relying solely on OpenAPI specifications for black-box REST API fuzzing, which often fails to provide sufficient test guidance, while existing supplementary data lacks a standardized, vendor-neutral format. To overcome this, the study introduces the first native integration of the OpenAPI Initiative (OAI) Overlay standard into the industrial-grade fuzzer EvoMaster, enabling structured injection of test example data via standardized Overlay files. This approach significantly enhances black-box testing capabilities without vendor lock-in and reduces cognitive overhead for testers. The effectiveness of the method is empirically validated on five real-world enterprise APIs, including two from Fortune 500 companies, demonstrating its practical utility and advancing the standardization of extensible techniques in REST API fuzzing.

Enterprises adopting commercial large language model (LLM) services face opaque cost structures: although input token count is controllable, output token volume is systematically influenced by user prompt linguistic style—particularly politeness—leading to unpredictable expenditure fluctuations that undermine budgeting and strategic decision-making. This study conducts controlled experiments via the OpenAI API, integrating NLP-based feature extraction with econometric analysis to uncover, for the first time, a dynamic, indirect mechanism whereby user language structure drives enterprise costs through the generative process. Results show that impolite prompts significantly increase output token count (mean +18.7%), raising costs and amplifying provider revenue without materially degrading response quality. These findings expose a structural flaw in prevailing token-based pricing models and provide empirical and theoretical foundations for designing more predictable, equitable LLM pricing mechanisms.

This work addresses the significant token redundancy in existing OpenAPI specifications when used in large language model (LLM) contexts, which hampers inference efficiency. To mitigate this, the authors propose LAPIS—a lightweight, LLM-optimized API specification that substantially reduces context length while preserving essential semantics. LAPIS achieves this through domain-specific constructs such as centralized error definitions, webhook trigger conditions, structured rate-limiting policies, and declarative operation workflows. Designed as a domain-specific language, LAPIS supports automatic conversion from OpenAPI 3.x without requiring a custom parser and remains compatible with the cl100k_base tokenizer. Experiments on five real-world APIs—including GitHub and Twilio—demonstrate that LAPIS reduces token consumption by 85.5% and 88.6% on average compared to OpenAPI YAML and JSON representations, respectively.

While existing OpenAPI specifications are structurally compliant, they often fail to meet the semantic requirements of AI agents for effective task planning and API invocation, leading to systematic failures. This work proposes Hermes, a multi-agent large language model system that pioneers the application of multi-agent LLMs to endpoint-level semantic readiness assessment of large-scale, production-grade APIs. Hermes analyzes OpenAPI specifications, identifies RESTful and documentation-related code smells, and generates interpretable diagnostic reports. Evaluated on 16 real-world APIs encompassing approximately 600 endpoints, Hermes detected 2,450 such smells, with high precision confirmed by developer validation. The findings have prompted enterprises to incorporate semantic documentation quality into their API governance frameworks and to redefine API design standards for AI agent compatibility.

Current RESTful API design quality assessment relies heavily on manual inspection, lacking early, automated validation mechanisms for non-functional requirements—particularly interoperability, modularity, and maintainability. Method: This paper proposes an OpenAPI-based static analysis approach that implements a configurable rule engine. It formalizes 75 design principles derived from scholarly literature and industry standards into structured, machine-checkable constraints, enabling customizable rule activation/deactivation and traceable feedback to align requirements engineering with architectural governance. Contribution/Results: Following the design science research paradigm, we developed and evaluated a prototype tool. Empirical evaluation and expert review demonstrate that the method significantly improves API design compliance and consistency, achieving 82% automation coverage. It effectively supports continuous architectural governance in agile development environments, bridging the gap between design-time assurance and operational API lifecycle management.