Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interaction

2025-10-23

AI Summary

This study addresses the limited contingency—i.e., timely, contextually appropriate, and semantically coherent response generation—in BabyLM for multi-turn dialogue. To this end, we propose ContingentChat: a teacher-student collaborative training paradigm; the first alignment dataset explicitly designed for response contingency; and an adaptive teacher decoding strategy for targeted post-training. Our model is built upon BabyLM, pretrained on a 100-million-word corpus. Experiments demonstrate substantial improvements in grammatical correctness and discourse coherence of generated responses. Although the adaptive decoding yields marginal gains, our results validate that contingency-oriented data construction and alignment training effectively enhance BabyLM’s dialogue capabilities. This work offers a novel pathway for low-resource, highly interactive language modeling, emphasizing pragmatic responsiveness over mere surface-level fluency.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Multi-turn dialogues between a child and a caregiver are characterized by a property called contingency - that is, prompt, direct, and meaningful exchanges between interlocutors. We introduce ContingentChat, a teacher-student framework that benchmarks and improves multi-turn contingency in a BabyLM trained on 100M words. Using a novel alignment dataset for post-training, BabyLM generates responses that are more grammatical and cohesive. Experiments with adaptive teacher decoding strategies show limited additional gains. ContingentChat demonstrates the benefits of targeted post-training for dialogue quality and indicates that contingency remains a challenging goal for BabyLMs.

Problem

• Improving multi-turn contingency in child-caregiver dialogues
• Enhancing grammaticality and cohesion in BabyLM responses
• Evaluating adaptive teacher strategies for dialogue quality

Innovation

• Teacher-student framework for multi-turn contingency
• Novel alignment dataset for post-training dialogue
• Adaptive teacher decoding strategies for limited gains