Towards Trustworthy Lexical Simplification: Exploring Safety and Efficiency with Small LLMs

2025-09-29

AI Summary

Large language models (LLMs) face critical security and deployment challenges in privacy-sensitive, resource-constrained settings—particularly for vulnerable users such as persons with disabilities—when applied to lexical simplification (LS). Method: We propose a localized, trustworthy LS framework based on small language models (SLMs), integrating synthetic-data-driven knowledge distillation, in-context learning, and a probability-guided harmful-output detection-and-filtering mechanism. Contribution/Results: We are the first to identify and empirically demonstrate that knowledge distillation can introduce safety risks in LS; accordingly, we construct the first SLM-oriented LS safety evaluation benchmark. Evaluated across five languages via both automated metrics and human assessment, our approach achieves a +32.7% improvement in safety (i.e., suppression of harmful simplifications) while preserving simplification quality (BLEU degradation <1.2). This work establishes a new paradigm for secure, efficient, and deployable multilingual LS at the edge.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Security and Privacy: Large-scale security measurements

Abstract

Despite their strong performance, large language models (LLMs) face challenges in real-world application of lexical simplification (LS), particularly in privacy-sensitive and resource-constrained environments. Moreover, since vulnerable user groups (e.g., people with disabilities) are one of the key target groups of this technology, it is crucial to ensure the safety and correctness of the output of LS systems. To address these issues, we propose an efficient framework for LS systems that utilizes small LLMs deployable in local environments. Within this framework, we explore knowledge distillation with synthesized data and in-context learning as baselines. Our experiments in five languages evaluate model outputs both automatically and manually. Our manual analysis reveals that while knowledge distillation boosts automatic metric scores, it also introduces a safety trade-off by increasing harmful simplifications. Importantly, we find that the model's output probability is a useful signal for detecting harmful simplifications. Leveraging this, we propose a filtering strategy that suppresses harmful simplifications while largely preserving beneficial ones. This work establishes a benchmark for efficient and safe LS with small LLMs. It highlights the key trade-offs between performance, efficiency, and safety, and demonstrates a promising approach for safe real-world deployment.

Problem

• Enhancing safety and efficiency of lexical simplification systems
• Addressing privacy and resource constraints with small LLMs
• Mitigating harmful simplifications while preserving beneficial outputs

Innovation

• Utilizes small LLMs for local deployment efficiency
• Employs knowledge distillation with synthesized data training
• Implements probability-based filtering to suppress harmful outputs