A Comunication Framework for Compositional Generation

2025-01-31

AI Summary

This work investigates how machine learning models can autonomously develop compositional structure and compositional generalization—without human-provided prompts—to emulate human-like novel concept generation. To this end, we propose a self-supervised generative framework grounded in sender-receiver communication games—the first application of such games to generative compositional modeling. We establish theoretical desiderata balancing efficiency, unambiguity, and non-holism, and introduce approximate message entropy regularization to encourage compositional encoding in discrete representations. Our method employs an iterative learning protocol, a pre-trained encoder-decoder architecture, and discrete message representations, trained self-supervisedly on Shapes3D. Experiments demonstrate substantial improvements in reconstruction accuracy and compositional metrics (mCC, gCC), outperforming existing discrete-message approaches.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Generation
• Computer Vision: Generative Adversarial Networks (GANs) for Vision

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Compositionality and compositional generalization--the ability to understand novel combinations of known concepts--are central characteristics of human language and are hypothesized to be essential for human cognition. In machine learning, the emergence of this property has been studied in a communication game setting, where independent agents (a sender and a receiver) converge to a shared encoding policy from a set of states to a space of discrete messages, where the receiver can correctly reconstruct the states observed by the sender using only the sender's messages. The use of communication games in generation tasks is still largely unexplored, with recent methods for compositional generation focusing mainly on the use of supervised guidance (either through class labels or text). In this work, we take the first steps to fill this gap, and we present a self-supervised generative communication game-based framework for creating compositional encodings in learned representations from pre-trained encoder-decoder models. In an Iterated Learning (IL) protocol involving a sender and a receiver, we apply alternating pressures for compression and diversity of encoded discrete messages, so that the protocol converges to an efficient but unambiguous encoding. Approximate message entropy regularization is used to favor compositional encodings. Our framework is based on rigorous justifications and proofs of defining and balancing the concepts of Eficiency, Unambiguity and Non-Holisticity in encoding. We test our method on the compositional image dataset Shapes3D, demonstrating robust performance in both reconstruction and compositionality metrics, surpassing other tested discrete message frameworks.

Problem

• Machine Learning
• Compositionality
• Creativity

Innovation

• Self-supervised Learning
• Compositional Encoding
• Communication Game Framework