failure-mode characterization

This study addresses the lack of a systematic taxonomy for failure modes in multi-provider large language model (LLM) service gateways, which hinders effective detection and diagnosis in production environments. The work proposes the first dual-axis structured failure classification framework for LLM gateways, categorizing failures by origin layer—spanning network/transport, streaming/protocol, state/session, model behavior, and governance/cost—and by detectability (explicit vs. implicit). Through root cause analysis, stress testing, mining of public bug reports, and protocol-level debugging, the authors construct a catalog of five validated failure cases, three of which include reproducible scripts. Notably, the study uncovers two previously undocumented silent failures: conversation history loss due to concurrency races and stream-index collisions corrupting tool-call payloads. These return HTTP 200 responses and pass standard health checks, evading detection without semantic-level observability, thereby posing significant threats to application reliability.

This paper addresses the under-recognized reliability challenges of large language models (LLMs) in real-world system deployments. Adopting a systems engineering perspective, it establishes the first fault taxonomy for LLM-based applications. Through systematic analysis and multi-case root-cause investigation, the study identifies 15 classes of latent failures—including multi-step reasoning drift, context boundary degradation, erroneous tool invocation, and latent inconsistency—exposing fundamental limitations of current evaluation benchmarks in stability, reproducibility, and workflow integration. The work introduces high-level design principles centered on observability, cost sensitivity, and version evolution, shifting LLM reliability research from a model-centric to a system-integration paradigm. It delivers the first structured fault classification framework and practical guidance for building reliable, maintainable, and auditable LLM software systems.

This study addresses safety risks arising from software faults in cyber-physical systems (CPS) for electric bicycles. We propose a simulation-driven functional Failure Mode and Effects Analysis (FMEA) method, leveraging Simulink Fault Analyzer to construct fault models, integrated with expert review and a systematic FMEA process to close the loop among fault modeling, simulation-based analysis, and impact assessment. Experimental evaluation identified 13 real-world faults with 100% model accuracy; among them, five revealed previously unrecognized safety implications, and 38.4% induced anomalous system behavior. The study distills ten reusable engineering practice guidelines, significantly enhancing the effectiveness and practicality of FMEA in industrial-scale CPS. It provides empirical validation and methodological contributions toward the operational deployment of simulation-driven safety analysis.

Traditional Failure Mode and Effects Analysis (FMEA) development for industrial equipment is highly manual, time-consuming, and suffers from low knowledge reuse. Method: This paper proposes a foundation-model-based approach for automated FMEA generation and structured database ingestion. It integrates domain-adapted natural language processing and information extraction to accurately identify fault modes, effects, causes, and detection mechanisms from unstructured technical documents, mapping them to standardized FMEA table entries. An interpretable, interactive correction mechanism enables expert feedback integration for iterative refinement, while structured outputs are automatically persisted into a relational database. Contribution/Results: Experiments demonstrate over 80% reduction in FMEA development cycle time, significantly improving efficiency and consistency in industrial asset knowledge construction. The approach validates the feasibility and practical value of foundation models in high-reliability industrial knowledge engineering applications.

This paper addresses the insufficient integration of early-system safety analysis with Model-Based Systems Engineering (MBSE). It comparatively evaluates three functional safety analysis methods—Failure Mode and Effects Analysis (FMEA), Functional Hazard Assessment (FHA), and Fault Feedback and Impact Propagation (FFIP)—and identifies FFIP as superior for detecting emergent behaviors, second-order effects, and fault propagation. Subsequently, it systematically reviews existing MBSE integration practices, categorizing them into four approaches: model transformation, custom algorithm development, built-in toolkits, and manual modeling. The study reveals that current integration efforts are predominantly focused on FMEA, while FHA and FFIP remain in exploratory stages, hindered by the absence of a unified framework and standardized guidelines. To bridge this gap, the paper proposes a novel, full-lifecycle safety analysis integration paradigm aligned with digital engineering transformation—enabling traceable, executable, and evolvable model-driven safety verification.

Naval systems frequently exhibit anomalous behaviors due to wear, misuse, or component failures—challenges that hinder timely detection and precise remediation. To address this, we propose a predictive-diagnostic closed-loop framework that tightly integrates the existing failure prediction system PREVENT with a newly designed responsive troubleshooting module, REACT. Methodologically, the framework synergizes multi-source time-series anomaly detection with domain-knowledge-driven fault-isolation process modeling, enabling end-to-end automation—from anomaly alerting and root-cause localization to actionable remediation recommendations. Evaluated on operational shipboard systems deployed by Fincantieri, the framework reduces mean time to fault localization by 42%, significantly improves operational response efficiency, and demonstrates strong generalizability across diverse industrial domains.

Current cyber-physical systems (CPS) in vehicular environments lack quantitative, experimentally grounded methods for assessing network resilience. Method: This study constructs an experimental testbed replicating real-world truck operational conditions and conducts multiple rounds of malware injection attacks, simultaneously collecting network- and physical-layer data on resistance and recovery behaviors. Contribution/Results: We introduce the novel concept of “bonware” to holistically characterize both cybersecurity defense capability and physical resilience, formalized via an analytically tractable mathematical model. We further define and extract experimentally identifiable, quantitative resilience metrics—termed elastic features—for the first time. Sensitivity analysis confirms these metrics exhibit significant discriminability with respect to attack intensity, defensive strategies, and physical redundancy. This work bridges a critical gap by advancing vehicular CPS resilience from qualitative description to quantifiable, comparable, and optimizable measurement.

This work addresses the limitations of traditional Failure Modes, Effects, and Diagnostic Analysis (FMEDA) in automotive ASIC functional safety verification, where expert judgment is used to estimate failure mode distributions and diagnostic coverage without quantifying associated uncertainties, thereby compromising reliability. For the first time, error propagation theory is systematically integrated into FMEDA to construct uncertainty models for both failure mode distributions and diagnostic coverage. This enables quantitative computation of the maximum deviations and confidence intervals for the Single-Point Fault Metric (SPFM) and Latent Fault Metric (LFM). Furthermore, an Error Importance Indicator (EII) is introduced to trace the key contributors driving overall uncertainty. The proposed approach significantly enhances the transparency and credibility of FMEDA, offering a scientifically rigorous and quantifiable foundation for compliance with ISO 26262.

This work addresses the limitations of generic pretrained models in industrial-scale video and live-stream content moderation, where platform-specific data distributions, policy objectives, and safety constraints are poorly aligned with off-the-shelf solutions, and systematic failure diagnosis and remediation mechanisms are lacking. The paper introduces a diagnostic methodology for audio-visual language models (AVLMs) that pioneers the characterization of model failures through observable feature signatures and establishes a principled mapping between failure categories and targeted intervention strategies, replacing heuristic trial-and-error approaches. Built upon multimodal foundation model architectures and validated on real-world platform traffic, this framework enables precise interventions throughout the model lifecycle. The resulting AVLM system has been deployed across more than 100 regions globally, significantly improving moderation accuracy and traceability for high-noise, semantically ambiguous, and highly diverse content.

This study addresses the challenge of effectively monitoring early-stage agent systems, where structural flaws often obscure task-level errors. The authors propose a three-dimensional (quality, suitability, efficiency) and three-granularity (intra-run, inter-run, structural) monitoring and triaging framework tailored for low-maturity agent systems. They introduce a novel system maturity staging model based on the coefficient of variation and monitoring granularity, integrated with a severity classification adapted from FMEA to guide human review. The resulting transferable monitoring architecture supports document-driven, multi-stage workflows, enhanced by a synthetic testbed with controlled error injection. Experimental results demonstrate that structural defects significantly mask task-level signals; 97% of issues can be automatically traced, with only 2% requiring human intervention, and each granularity level precisely identifies its corresponding defect type (coefficients of variation: 0.02, 1.25, and 0.00, respectively).

This study addresses the challenge of efficiently localizing faulty modules in automotive system-level 0D simulations following model updates—a process that traditionally incurs high verification costs and prolonged cycles. To overcome this, the authors propose a novel diagnostic approach based on graph-structured modeling, which uniquely integrates Dynamic Mode Decomposition (DMD), linear programming, and autoencoders to embed system simulation behaviors into a graph representation. This framework enables automatic fault module identification with only a minimal number of simulation runs. The proposed method substantially reduces computational overhead, enhances fault localization efficiency, and seamlessly integrates into existing engineering validation workflows, offering both practical utility and strong scalability.

This study addresses the challenge of quantifying the sensitivity of classification models to feature errors in training data by proposing the Error Sensitivity Profile (ESP)—a novel metric that systematically defines and measures the impact of single or multiple feature errors on model performance. Through experiments on two widely used datasets involving 14 classification models and leveraging a custom-developed dirty data toolkit, the authors demonstrate that model performance degradation is not necessarily correlated with simple feature–target variable associations. ESP effectively identifies the error types and critical features that most significantly impair predictive accuracy, thereby offering actionable guidance for prioritizing data cleaning efforts.