Once Upon a Time: Interactive Learning for Storytelling with Small Language Models

2025-09-19

AI Summary

This study addresses the challenge of reducing data requirements for small language models (SLMs) in narrative generation. Methodologically, it introduces a student–teacher collaborative architecture: a teacher model provides multidimensional automated feedback—assessing readability, narrative coherence, and creativity—while the student model jointly optimizes next-token prediction and feedback-driven reinforcement learning, emulating child-like interactive language acquisition. Experiments demonstrate that the student model achieves narrative performance comparable to that of a 410M-parameter language model trained solely on 4.1 million tokens, using only 1 million tokens of interactive feedback data. The core contribution is the first empirical validation that high-order semantic feedback enables substantial gains in functional linguistic capability for SLMs under extreme data scarcity. This work establishes a novel, efficient, and interpretable paradigm for training resource-constrained language models.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Children efficiently acquire language not just by listening, but by interacting with others in their social environment. Conversely, large language models are typically trained with next-word prediction on massive amounts of text. Motivated by this contrast, we investigate whether language models can be trained with less data by learning not only from next-word prediction but also from high-level, cognitively inspired feedback. We train a student model to generate stories, which a teacher model rates on readability, narrative coherence, and creativity. By varying the amount of pretraining before the feedback loop, we assess the impact of this interactive learning on formal and functional linguistic competence. We find that the high-level feedback is highly data efficient: With just 1 M words of input in interactive learning, storytelling skills can improve as much as with 410 M words of next-word prediction.

Problem

• Investigating data-efficient training for language models
• Using interactive feedback to improve storytelling quality
• Assessing impact of high-level feedback on linguistic competence

Innovation

• Interactive learning with teacher feedback
• High-level cognitive feedback mechanisms
• Data-efficient storytelling model training