Search-based Testing of Simulink Models with Requirements Tables

2025-01-09

AI Summary

Existing search-based software testing (SBST) methods for Simulink models struggle to directly support natural-language Requirements Tables (RTs), necessitating cumbersome manual formalization of requirements into logical constraints. Method: This paper proposes the first black-box SBST framework natively driven by RTs. It introduces a semantic parsing and constraint mapping mechanism that automatically translates natural-language requirements in RTs into executable test constraints, integrated with genetic algorithms and Simulink’s simulation interface for end-to-end automated test generation—bypassing explicit logical formula translation. Contribution/Results: Evaluated on 60 real-world model–RT pairs, the approach achieves a 70% failure-revealing test case generation rate, matching the efficiency of state-of-the-art non-RT-based SBST tools. Moreover, it uncovered three critical failures in a cruise control model missed by other tools, demonstrating both industrial applicability and technical novelty.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data generation and simulation
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Search-based software testing (SBST) of Simulink models helps find scenarios that demonstrate that the system can reach a state that violates one of its requirements. However, many SBST techniques for Simulink models rely on requirements being expressed in logical languages, limiting their adoption in industry. To help with the adoption, SBST methods and tools for Simulink models need to be integrated with tools used by engineers to specify requirements. This work presents the first black-box testing approach for Simulink models that supports Requirements Table (RT), a tool from Simulink Requirements Toolbox used by practitioners to express software requirements. We evaluated our solution by considering 60 model-RT combinations each made by a model and an RT. Our SBST framework returned a failure-revealing test case for 70% of the model-RT combinations. Remarkably, it identified a failure-revealing test case for three model-RT combinations for a cruise controller of an industrial simulator that other previously used tools were not able to find. The efficiency of our SBST solution is acceptable for practical applications and comparable with existing SBST tools that are not based on RT.

Problem

• Search-Based Software Testing
• Simulink Requirements Tables
• Violation Scenarios Detection

Innovation

• Search-Based Software Testing
• Simulink Requirement Tables
• Practical Efficiency