Image Generators are Generalist Vision Learners

2026-04-22

AI Summary

This study investigates whether image generation models can acquire general-purpose visual understanding through generative pretraining. The authors reformulate diverse vision tasks as image generation problems by unifying their outputs into RGB images. Building upon the generatively pretrained model Nano Banana Pro, they perform lightweight instruction tuning using a combination of the original generative data and minimal task-specific data to develop Vision Banana, a unified vision model. This work provides the first systematic validation that generative pretraining on images can serve as a viable paradigm for general visual learning. Vision Banana achieves state-of-the-art zero-shot performance—matching or surpassing specialized models—on a range of 2D and 3D vision tasks, including segmentation and depth estimation, while preserving its original image generation capabilities.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Recent works show that image and video generators exhibit zero-shot visual understanding behaviors, in a way reminiscent of how LLMs develop emergent capabilities of language understanding and reasoning from generative pretraining. While it has long been conjectured that the ability to create visual content implies an ability to understand it, there has been limited evidence that generative vision models have developed strong understanding capabilities. In this work, we demonstrate that image generation training serves a role similar to LLM pretraining, and lets models learn powerful and general visual representations that enable SOTA performance on various vision tasks. We introduce Vision Banana, a generalist model built by instruction-tuning Nano Banana Pro (NBP) on a mixture of its original training data alongside a small amount of vision task data. By parameterizing the output space of vision tasks as RGB images, we seamlessly reframe perception as image generation. Our generalist model, Vision Banana, achieves SOTA results on a variety of vision tasks involving both 2D and 3D understanding, beating or rivaling zero-shot domain-specialists, including Segment Anything Model 3 on segmentation tasks, and the Depth Anything series on metric depth estimation. We show that these results can be achieved with lightweight instruction-tuning without sacrificing the base model's image generation capabilities. The superior results suggest that image generation pretraining is a generalist vision learner. It also shows that image generation serves as a unified and universal interface for vision tasks, similar to text generation's role in language understanding and reasoning. We could be witnessing a major paradigm shift for computer vision, where generative vision pretraining takes a central role in building Foundational Vision Models for both generation and understanding.

Problem

• image generation
• vision understanding
• generalist vision learner
• foundational vision models
• zero-shot visual tasks

Innovation

• image generation pretraining
• generalist vision learner
• vision as image generation
• instruction-tuning
• foundational vision models