GenVP: Generating Visual Puzzles with Contrastive Hierarchical VAEs

2025-03-30

AI Summary

Abstract Visual Reasoning (AVR) models for Raven’s Progressive Matrices (RPM) suffer from a unidirectional bottleneck—while excelling at puzzle solving, they lack the capability to generate novel, rule-compliant RPM instances. Method: We propose the first systematic framework for controllable RPM generation, introducing a contrastive hierarchical Variational Autoencoder (VAE). Our architecture explicitly disentangles abstract relational rules from visual attributes via hierarchical latent representations, integrating contrastive learning, structured latent disentanglement, and out-of-distribution (OOD) robust training. Contribution/Results: The model achieves bidirectional AVR: it matches state-of-the-art (SOTA) solving accuracy across five benchmark datasets while enabling rule-guided puzzle generation and multi-solution prompt completion. It attains superior performance across 22 OOD generalization scenarios and maintains high efficiency and generative diversity even under expanded solution-space constraints.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Reasoning under Uncertainty: Relational Probabilistic Models
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Raven's Progressive Matrices (RPMs) is an established benchmark to examine the ability to perform high-level abstract visual reasoning (AVR). Despite the current success of algorithms that solve this task, humans can generalize beyond a given puzzle and create new puzzles given a set of rules, whereas machines remain locked in solving a fixed puzzle from a curated choice list. We propose Generative Visual Puzzles (GenVP), a framework to model the entire RPM generation process, a substantially more challenging task. Our model's capability spans from generating multiple solutions for one specific problem prompt to creating complete new puzzles out of the desired set of rules. Experiments on five different datasets indicate that GenVP achieves state-of-the-art (SOTA) performance both in puzzle-solving accuracy and out-of-distribution (OOD) generalization in 22 OOD scenarios. Compared to SOTA generative approaches, which struggle to solve RPMs when the feasible solution space increases, GenVP efficiently generalizes to these challenging setups. Moreover, our model demonstrates the ability to produce a wide range of complete RPMs given a set of abstract rules by effectively capturing the relationships between abstract rules and visual object properties.

Problem

• Generating new visual puzzles beyond fixed datasets
• Modeling abstract rules for visual puzzle creation
• Improving generalization in visual reasoning tasks

Innovation

• Contrastive Hierarchical VAEs for puzzle generation
• Generates multiple solutions from problem prompts
• Creates new puzzles from abstract rules