Large Language Models are Good Multi-lingual Learners : When LLMs Meet Cross-lingual Prompts

2024-09-17

International Conference on Computational Linguistics

AI Summary

Large language models (LLMs) frequently overlook critical constraints during long-context reasoning, leading to rule violation. To address this, we propose MLPrompt—a cross-lingual prompting strategy that automatically translates error-prone rules into multiple linguistic formulations to enhance model attention to key constraints. MLPrompt introduces the novel prompting paradigm of “multilingual translation for reinforced rule awareness,” integrated with an automated verification mechanism, and supports structured generation tasks including mixed-integer programming (MIP) modeling and text-to-SQL. Experiments across multiple public benchmark datasets demonstrate that MLPrompt significantly outperforms Chain-of-Thought, Tree-of-Thought, and Self-Consistency baselines. In text-to-MIP and text-to-SQL tasks, it yields more robust and accurate structured outputs. These results empirically validate the efficacy of multilingual prompting in improving rule-aware modeling for complex constrained reasoning.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Prompt Engineering / Prompting
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

With the advent of Large Language Models (LLMs), generating rule-based data for real-world applications has become more accessible. Due to the inherent ambiguity of natural language and the complexity of rule sets, especially in long contexts, LLMs often struggle to follow all specified rules, frequently omitting at least one. To enhance the reasoning and understanding of LLMs on long and complex contexts, we propose a novel prompting strategy Multi-Lingual Prompt, namely MLPrompt, which automatically translates the error-prone rule that an LLM struggles to follow into another language, thus drawing greater attention to it. Experimental results on public datasets across various tasks have shown MLPrompt can outperform state-of-the-art prompting methods such as Chain of Thought, Tree of Thought, and Self-Consistency. Additionally, we introduce a framework integrating MLPrompt with an auto-checking mechanism for structured data generation, with a specific case study in text-to-MIP instances. Further, we extend the proposed framework for text-to-SQL to demonstrate its generation ability towards structured data synthesis.

Problem

• Improves LLM rule-following in complex multilingual contexts
• Enhances structured data generation with auto-checking mechanism
• Outperforms existing prompting methods in cross-lingual tasks

Innovation

• MLPrompt translates rules into another language
• Integrates auto-checking for structured data generation
• Extends framework for text-to-SQL synthesis