Problem
Research questions and friction points this paper is trying to address.
Few-shot Learning
Abstract Concept Recognition
Real-world Image Analysis
Bongard-OpenWorld: Few-Shot Reasoning for Free-form Visual Concepts in the Real World
🤖 AI Summary
This work addresses few-shot visual concept reasoning: given only a few real-world positive examples, models must induce open-vocabulary compositional visual concepts (e.g., “a striped cat sitting on a windowsill”) and determine whether novel images satisfy them. To this end, we introduce the first benchmark for few-shot visual concept reasoning on real images—extending classical Bongard problems along three dimensions: open-world settings, free-form concept definitions, and photorealistic inputs. We propose a human-in-the-loop neuro-symbolic reasoning framework that tightly integrates vision-language models (VLMs), large language models (LLMs), and formal logical inference. Empirical evaluation reveals that state-of-the-art models achieve only 64% accuracy—substantially below human performance (91%)—highlighting fundamental limitations in current visual intelligence regarding abstract concept induction and commonsense integration.
📝 Abstract
We introduce Bongard-OpenWorld, a new benchmark for evaluating real-world few-shot reasoning for machine vision. It originates from the classical Bongard Problems (BPs): Given two sets of images (positive and negative), the model needs to identify the set that query images belong to by inducing the visual concepts, which is exclusively depicted by images from the positive set. Our benchmark inherits the few-shot concept induction of the original BPs while adding the two novel layers of challenge: 1) open-world free-form concepts, as the visual concepts in Bongard-OpenWorld are unique compositions of terms from an open vocabulary, ranging from object categories to abstract visual attributes and commonsense factual knowledge; 2) real-world images, as opposed to the synthetic diagrams used by many counterparts. In our exploration, Bongard-OpenWorld already imposes a significant challenge to current few-shot reasoning algorithms. We further investigate to which extent the recently introduced Large Language Models (LLMs) and Vision-Language Models (VLMs) can solve our task, by directly probing VLMs, and combining VLMs and LLMs in an interactive reasoning scheme. We even conceived a neuro-symbolic reasoning approach that reconciles LLMs&VLMs with logical reasoning to emulate the human problem-solving process for Bongard Problems. However, none of these approaches manage to close the human-machine gap, as the best learner achieves 64% accuracy while human participants easily reach 91%. We hope Bongard-OpenWorld can help us better understand the limitations of current visual intelligence and facilitate future research on visual agents with stronger few-shot visual reasoning capabilities.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Bongard-OpenWorld
open vocabulary
human-like reasoning
🔎 Similar Papers
No similar papers found.
