Towards Compositional Generalization of LLMs via Skill Taxonomy Guided Data Synthesis

2026-01-07

arXiv.org

AI Summary

This work addresses the limitations of large language models in compositional generalization, which stem from the long-tailed distribution of complex skill combinations in training data, leading to insufficient instruction-following and agent task generalization. To overcome this, the authors propose the STEPS framework, which uniquely integrates hierarchical skill structure with information maximization. Leveraging structural information theory, STEPS constructs a hierarchical skill taxonomy and formulates data synthesis as a constrained information maximization problem, generating training examples that are both semantically consistent and compositionally challenging. This approach enables interpretable and systematic compositional data synthesis, significantly outperforming existing data augmentation methods across multiple instruction-following benchmarks and downstream agent tasks, thereby enhancing the model’s compositional generalization capabilities.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Learning to Search
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Large Language Models (LLMs) and agent-based systems often struggle with compositional generalization due to a data bottleneck in which complex skill combinations follow a long-tailed, power-law distribution, limiting both instruction-following performance and generalization in agent-centric tasks. To address this challenge, we propose STEPS, a Skill Taxonomy guided Entropy-based Post-training data Synthesis framework for generating compositionally challenging data. STEPS explicitly targets compositional generalization by uncovering latent relationships among skills and organizing them into an interpretable, hierarchical skill taxonomy using structural information theory. Building on this taxonomy, we formulate data synthesis as a constrained information maximization problem, selecting skill combinations that maximize marginal structural information within the hierarchy while preserving semantic coherence. Experiments on challenging instruction-following benchmarks show that STEPS outperforms existing data synthesis baselines, while also yielding improved compositional generalization in downstream agent-based evaluations.

Problem

• compositional generalization
• data bottleneck
• skill combinations
• long-tailed distribution
• instruction-following

Innovation

• compositional generalization
• skill taxonomy
• data synthesis
• information maximization
• structural information theory