ClarifySTL: An Interactive LLM Agent Framework for STL Transformation through Requirements Clarification

2026-05-01

AI Summary

Natural language requirements often contain vagueness and ambiguity, making it challenging to accurately translate them into Signal Temporal Logic (STL) specifications. This work proposes ClarifySTL, a novel framework that introduces an interactive large language model agent to address this issue. The agent iteratively detects ambiguous or vague expressions, generates targeted clarification questions, incorporates user feedback, and refines the STL formula to faithfully capture the user’s intent. Evaluated on established benchmarks DeepSTL and STL-DivEn, as well as a newly introduced AmbiEval dataset, ClarifySTL significantly outperforms existing non-interactive approaches, demonstrating marked improvements in both the accuracy of formal specifications and user-friendliness.

Abstract

Signal Temporal Logic (STL) is a formal language for specifying real-time behaviors of cyber-physical systems (CPS). Automatically transforming natural language requirements into STL specifications has received growing attention. Recent efforts leveraging large language models (LLMs) have demonstrated impressive performance, but some natural language requirements in practice contain vague or ambiguous information, which remains challenging for LLMs to handle. To address these challenges, we propose ClarifySTL, an interactive LLM-agent framework that enhances STL transformation through requirements clarification. ClarifySTL first detects vague expressions that indicate underspecified information in a requirement. If any vagueness is detected, it generates targeted clarification queries to guide users in supplementing the requirement until all necessary details are provided. Subsequently, if ClarifySTL detects ambiguities, it formulates focused ambiguity clarification queries and updates the requirements based on user feedback until all ambiguities are resolved. Finally, the requirements with vagueness and ambiguity clarified are transformed into STL specifications using LLMs. This interactive framework ensures that the resulting STL formulas faithfully capture user intent while reducing the burden on the user. We evaluate ClarifySTL on the representative benchmarks DeepSTL and STL-DivEn, as well as our newly introduced AmbiEval benchmark, which is specifically designed to assess the performance of the agents in handling vagueness and ambiguity, including both detection and query generation. The experimental results show that ClarifySTL is effective.

Problem

• Signal Temporal Logic
• natural language requirements
• vagueness
• ambiguity
• requirements clarification

Innovation

• interactive LLM agent
• requirements clarification
• Signal Temporal Logic (STL)
• vagueness detection
• ambiguity resolution