Investigating the Influence of Prompt and Response Languages on LLM Content Generation

2026-08-22

AI Summary

研究探讨了提示语和回答语言对大型语言模型内容生成的影响，通过五种模型在四种语言条件下评估了68个非翻译问题的答案，分析了响应长度、语义保真度及跨语言关键词重叠。

Abstract

This study examines how prompt and response language influence the behavior of large language models. Using five models, we evaluated answers to 68 non translation questions across four language conditions: English to English, English to Norwegian, Norwegian to Norwegian, and Norwegian to English. After removing refused items, the dataset contains 1348 responses. We measure length differences with Cohen d, semantic fidelity with LabSE cosine similarity, and cross lingual keyword overlap with both raw and soft Jaccard. Prompt language has a strong effect on response length. With English output, Norwegian prompts shorten responses by about thirty seven percent. With Norwegian output, English prompts shorten responses by about forty one percent. The largest cross lingual contrast shows a reduction in word count but a smaller reduction in tokens, reflecting tokenizer differences. Despite variation in length, semantic similarity remains high, and soft Jaccard reveals substantial conceptual overlap that raw Jaccard does not capture. Effect sizes vary across models, indicating heterogeneity. Prompt language is not neutral and systematically shapes output length and lexical realization, with implications for multilingual prompt design.

Problem

• prompt language
• response language
• large language models
• semantic fidelity
• cross-lingual keyword overlap

Innovation

• Prompt Language
• Response Length
• Semantic Fidelity
• Cross-lingual Keyword Overlap
• Soft Jaccard