Post-training for Efficient Communication via Convention Formation

2025-08-08

AI Summary

Large language models (LLMs) lack human-like dynamic linguistic adaptation and emergent convention formation during multi-turn interactions, hindering efficient collaborative reasoning. To address this, we propose a cognition-driven post-training paradigm that employs heuristically constructed demonstrations of convention formation for supervised fine-tuning, thereby eliciting the model’s capacity to self-adaptively evolve communication strategies. To rigorously evaluate this capability, we design and release two novel benchmarks: a cognitive reasoning–oriented convention formation benchmark and a document-grounded convention formation benchmark. Experimental results demonstrate that our method significantly improves convention formation performance across both benchmarks, with behavioral patterns closely approximating human conventions. This work provides the first empirical evidence that post-training can endow LLMs with human-like, dynamic convention generation capabilities—marking a critical step toward cognitively grounded, collaborative AI systems.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

Humans communicate with increasing efficiency in multi-turn interactions, by adapting their language and forming ad-hoc conventions. In contrast, prior work shows that LLMs do not naturally show this behavior. We develop a post-training process to develop this ability through targeted fine-tuning on heuristically identified demonstrations of convention formation. We evaluate with two new benchmarks focused on this capability. First, we design a focused, cognitively-motivated interaction benchmark that consistently elicits strong convention formation trends in humans. Second, we create a new document-grounded reference completion task that reflects in-the-wild convention formation behavior. Our studies show significantly improved convention formation abilities in post-trained LLMs across the two evaluation methods.

Problem

• Enhancing LLMs' efficiency in multi-turn communication
• Developing post-training for convention formation
• Evaluating with new benchmarks for convention behavior

Innovation

• Post-training process for convention formation
• Heuristically identified fine-tuning demonstrations
• New benchmarks for evaluating convention formation