Evaluation of Multilingual LLMs Personalized Text Generation Capabilities Targeting Groups and Social-Media Platforms

2026-01-07

arXiv.org

AI Summary

This study addresses the underexplored risk that multilingual large language models (LLMs) may be exploited for cross-lingual, targeted disinformation through personalized text generation, with limited systematic evidence on how such personalization affects the detectability of machine-generated content across languages. For the first time, we systematically evaluate the personalization capabilities of 16 LLMs across 10 languages, generating 17,280 texts from 1,080 prompt combinations tailored to demographic attributes and social media platforms. Through cross-lingual controlled experiments integrating text quality assessment and detection techniques, we find that platform-specific targeting significantly reduces detectability more than demographic targeting, with this effect most pronounced in English. Our results reveal interlingual variations and highlight the heightened stealth risks associated with platform-oriented generation.

Technology Category

• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Language and Vision

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Capabilities of large language models to generate multilingual coherent text have continuously enhanced in recent years, which opens concerns about their potential misuse. Previous research has shown that they can be misused for generation of personalized disinformation in multiple languages. It has also been observed that personalization negatively affects detectability of machine-generated texts; however, this has been studied in the English language only. In this work, we examine this phenomenon across 10 languages, while we focus not only on potential misuse of personalization capabilities, but also on potential benefits they offer. Overall, we cover 1080 combinations of various personalization aspects in the prompts, for which the texts are generated by 16 distinct language models (17,280 texts in total). Our results indicate that there are differences in personalization quality of the generated texts when targeting demographic groups and when targeting social-media platforms across languages. Personalization towards platforms affects detectability of the generated texts in a higher scale, especially in English, where the personalization quality is the highest.

Problem

• multilingual LLMs
• personalized text generation
• disinformation
• detectability
• social-media platforms

Innovation

• multilingual LLMs
• personalized text generation
• detectability
• cross-lingual evaluation
• social-media targeting