Learning to Learn from Language Feedback with Social Meta-Learning

2026-02-18

AI Summary

This work addresses the limited adaptability of large language models (LLMs) in dialogue settings, where they often fail to proactively solicit and effectively utilize linguistic feedback, resulting in rigid and unresponsive interactions. Inspired by human social meta-learning, the study formalizes social meta-learning as a novel fine-tuning paradigm for LLMs. By constructing simulated pedagogical dialogues and modeling multi-turn interactive tasks, the approach enables models to actively seek and leverage natural language feedback. Empirical results demonstrate that this method significantly enhances the model’s ability to handle ambiguous or information-scarce queries across diverse domains such as mathematics and programming, reduces premature answering, and improves problem-solving performance over extended multi-turn interactions.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) often struggle to learn from corrective feedback within a conversational context. They are rarely proactive in soliciting this feedback, even when faced with ambiguity, which can make their dialogues feel static, one-sided, and lacking the adaptive qualities of human conversation. To address these limitations, we draw inspiration from social meta-learning (SML) in humans - the process of learning how to learn from others. We formulate SML as a finetuning methodology, training LLMs to solicit and learn from language feedback in simulated pedagogical dialogues, where static tasks are converted into interactive social learning problems. SML effectively teaches models to use conversation to solve problems they are unable to solve in a single turn. This capability generalises across domains; SML on math problems produces models that better use feedback to solve coding problems and vice versa. Furthermore, despite being trained only on fully-specified problems, these models are better able to solve underspecified tasks where critical information is revealed over multiple turns. When faced with this ambiguity, SML-trained models make fewer premature answer attempts and are more likely to ask for the information they need. This work presents a scalable approach to developing AI systems that effectively learn from language feedback.

Problem

• language feedback
• conversational learning
• ambiguity handling
• interactive dialogue
• social learning

Innovation

• Social Meta-Learning
• Language Feedback
• Interactive Learning
• Large Language Models
• Pedagogical Dialogue