GenAI-based test case generation and execution in SDV platform

2025-09-05

AI Summary

To address low efficiency in manual test case authoring, poor cross-subsystem compatibility, and difficulties integrating third-party tools in Software-Defined Vehicle (SDV) platforms, this paper proposes a generative AI–based method for automated, structured test case generation. The approach synergistically integrates Large Language Models (LLMs) and Vision-Language Models (VLMs) to jointly parse natural-language requirements and system architecture diagrams, while leveraging Vehicle Signal Specification (VSS)-based signal-level modeling to generate executable Gherkin-compliant test cases. End-to-end automated validation is performed in the open, vendor-neutral digital.auto environment. Key contributions include: (i) the first application of VLMs to SDV test case generation, enabling multimodal requirement understanding; (ii) a standardized, signal-level modeling framework enhancing cross-domain interoperability; and (iii) seamless integration with mainstream testing toolchains. Evaluation on a child-presence detection system demonstrates an ~80% improvement in test case generation efficiency and >50% reduction in manual intervention.

Technology Category

• Computer Vision: Large Vision Models
• Planning, Routing, and Scheduling: Planning with Language Models
• Natural Language Processing: Generation

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

This paper introduces a GenAI-driven approach for automated test case generation, leveraging Large Language Models and Vision-Language Models to translate natural language requirements and system diagrams into structured Gherkin test cases. The methodology integrates Vehicle Signal Specification modeling to standardize vehicle signal definitions, improve compatibility across automotive subsystems, and streamline integration with third-party testing tools. Generated test cases are executed within the digital.auto playground, an open and vendor-neutral environment designed to facilitate rapid validation of software-defined vehicle functionalities. We evaluate our approach using the Child Presence Detection System use case, demonstrating substantial reductions in manual test specification effort and rapid execution of generated tests. Despite significant automation, the generation of test cases and test scripts still requires manual intervention due to current limitations in the GenAI pipeline and constraints of the digital.auto platform.

Problem

• Automating test case generation from natural language and diagrams
• Standardizing vehicle signal definitions for improved compatibility
• Reducing manual effort in software-defined vehicle validation

Innovation

• Leverages LLMs and VLMs for test case generation
• Integrates Vehicle Signal Specification modeling
• Executes tests in vendor-neutral digital.auto playground