GENIUS: Generative Fluid Intelligence Evaluation Suite

2026-02-11

AI Summary

This work addresses a critical gap in current evaluation benchmarks, which predominantly emphasize crystallized intelligence while neglecting generative fluid intelligence (GFI)—the ability of models to infer patterns, reason about constraints, and adapt instantaneously in novel contexts. The study formally defines GFI for the first time and introduces GENIUS, a comprehensive evaluation suite that systematically assesses unified multimodal models across three core dimensions: inducing implicit patterns, executing ad-hoc constraints, and adapting to contextual knowledge. Through context-driven task design and attention mechanism analysis, evaluations of twelve leading models reveal that their GFI limitations primarily stem from inadequate contextual understanding. Building on this insight, the authors propose a training-free attention intervention strategy that significantly enhances model performance, thereby validating the efficacy of their approach.

Technology Category

• Natural Language Processing: Generation
• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Unified Multimodal Models (UMMs) have shown remarkable progress in visual generation. Yet, existing benchmarks predominantly assess $\textit{Crystallized Intelligence}$, which relies on recalling accumulated knowledge and learned schemas. This focus overlooks $\textit{Generative Fluid Intelligence (GFI)}$: the capacity to induce patterns, reason through constraints, and adapt to novel scenarios on the fly. To rigorously assess this capability, we introduce $\textbf{GENIUS}$ ($\textbf{GEN}$ Fluid $\textbf{I}$ntelligence Eval$\textbf{U}$ation $\textbf{S}$uite). We formalize $\textit{GFI}$ as a synthesis of three primitives. These include $\textit{Inducing Implicit Patterns}$ (e.g., inferring personalized visual preferences), $\textit{Executing Ad-hoc Constraints}$ (e.g., visualizing abstract metaphors), and $\textit{Adapting to Contextual Knowledge}$ (e.g., simulating counter-intuitive physics). Collectively, these primitives challenge models to solve problems grounded entirely in the immediate context. Our systematic evaluation of 12 representative models reveals significant performance deficits in these tasks. Crucially, our diagnostic analysis disentangles these failure modes. It demonstrates that deficits stem from limited context comprehension rather than insufficient intrinsic generative capability. To bridge this gap, we propose a training-free attention intervention strategy. Ultimately, $\textbf{GENIUS}$ establishes a rigorous standard for $\textit{GFI}$, guiding the field beyond knowledge utilization toward dynamic, general-purpose reasoning. Our dataset and code will be released at: $\href{https://github.com/arctanxarc/GENIUS}{https://github.com/arctanxarc/GENIUS}$.

Problem

• Generative Fluid Intelligence
• Crystallized Intelligence
• Multimodal Models
• Contextual Reasoning
• Visual Generation

Innovation

• Generative Fluid Intelligence
• Contextual Reasoning
• Multimodal Generation
• Attention Intervention
• Implicit Pattern Induction