Lumos: Let there be Language Model System Certification

2025-12-02

AI Summary

This work addresses the critical lack of scalable, formal behavioral specifications and certification frameworks for language model systems (LMS). We propose Lumos, the first systematic framework enabling formal safety certification of LMS. Lumos models stochastic prompt distributions as directed graphs, integrates an imperative probabilistic programming DSL, subgraph sampling, and denotational semantics to precisely specify complex relational and temporal properties, and couples these with statistical verification tools for rigorous safety validation. Notably, Lumos introduces the first modular, scalable safety specification framework for vision-language models in autonomous driving scenarios. Experimental evaluation reveals that Qwen-VL exhibits over 90% probability of generating unsafe outputs during right-turn maneuvers in rainy conditions; Lumos successfully identifies and localizes such failure cases, exposing severe safety vulnerabilities in current models.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We introduce the first principled framework, Lumos, for specifying and formally certifying Language Model System (LMS) behaviors. Lumos is an imperative probabilistic programming DSL over graphs, with constructs to generate independent and identically distributed prompts for LMS. It offers a structured view of prompt distributions via graphs, forming random prompts from sampled subgraphs. Lumos supports certifying LMS for arbitrary prompt distributions via integration with statistical certifiers. We provide hybrid (operational and denotational) semantics for Lumos, providing a rigorous way to interpret the specifications. Using only a small set of composable constructs, Lumos can encode existing LMS specifications, including complex relational and temporal specifications. It also facilitates specifying new properties - we present the first safety specifications for vision-language models (VLMs) in autonomous driving scenarios developed with Lumos. Using these, we show that the state-of-the-art VLM Qwen-VL exhibits critical safety failures, producing incorrect and unsafe responses with at least 90% probability in right-turn scenarios under rainy driving conditions, revealing substantial safety risks. Lumos's modular structure allows easy modification of the specifications, enabling LMS certification to stay abreast with the rapidly evolving threat landscape. We further demonstrate that specification programs written in Lumos enable finding specific failure cases exhibited by state-of-the-art LMS. Lumos is the first systematic and extensible language-based framework for specifying and certifying LMS behaviors, paving the way for a wider adoption of LMS certification.

Problem

• Develops a principled framework for specifying and certifying language model system behaviors
• Enables formal certification of language models for arbitrary prompt distributions
• Identifies critical safety failures in vision-language models for autonomous driving

Innovation

• Uses imperative probabilistic programming DSL over graphs
• Generates independent prompts via sampled subgraph distributions
• Integrates with statistical certifiers for arbitrary prompt distributions