PolyPrompt: Automating Knowledge Extraction from Multilingual Language Models with Dynamic Prompt Generation

2025-02-27

AI Summary

To address inconsistent cross-lingual performance of multilingual large language models (LLMs) on knowledge extraction tasks, this paper proposes a parameter-free, gradient-driven multilingual trigger word learning method. The approach dynamically identifies input language, optimizes lightweight language-adaptive triggers via gradient-based search, and automatically selects and injects them—without modifying any model parameters. Its core innovation is the first gradient-driven prompt generation mechanism, enabling efficient, language-aware prompt engineering. Evaluated on the MMLU benchmark across 15 languages, the method achieves accuracy improvements of 3.7%–19.9% on two ~1B-parameter models, significantly enhancing cross-lingual reasoning consistency. It establishes a scalable, low-overhead paradigm for multilingual prompt optimization.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Large language models (LLMs) showcase increasingly impressive English benchmark scores, however their performance profiles remain inconsistent across multilingual settings. To address this gap, we introduce PolyPrompt, a novel, parameter-efficient framework for enhancing the multilingual capabilities of LLMs. Our method learns a set of trigger tokens for each language through a gradient-based search, identifying the input query's language and selecting the corresponding trigger tokens which are prepended to the prompt during inference. We perform experiments on two ~1 billion parameter models, with evaluations on the global MMLU benchmark across fifteen typologically and resource diverse languages, demonstrating accuracy gains of 3.7%-19.9% compared to naive and translation-pipeline baselines.

Problem

• Enhancing multilingual capabilities of LLMs
• Dynamic prompt generation for language identification
• Improving accuracy across diverse languages

Innovation

• Dynamic prompt generation
• Gradient-based trigger tokens
• Multilingual accuracy enhancement