COCOLogic-V2: Identifying Logical Inconsistencies via Truly Hard-Negatives

2026-06-26

AI Summary

This work addresses the lack of effective evaluation of logical consistency in visual inductive reasoning by existing interpretable models on real-world images. The authors construct an object-centric dataset grounded in real images that covers a subset of first-order logic and introduce a novel fine-grained partitioning mechanism distinguishing near-boundary from far-boundary negative samples to enable precise diagnostic assessment of models’ logical reasoning capabilities. By integrating object-centric representations, first-order logic rule modeling, and hierarchical negative sampling, their evaluation framework reveals that while models effectively discriminate between positive and negative samples overall, performance degrades significantly on near-boundary negatives. Furthermore, perceptual noise and an expanded rule search space exacerbate reasoning difficulties, particularly in few-shot scenarios.

Abstract

While interpretable models such as concept bottleneck models (CBMs) and program synthesis methods enable verification of model decisions, their evaluation is typically limited to simple tasks, leaving complex reasoning on real-world images largely unexplored. We introduce COCOLogic-V2, an object-centric dataset for visual inductive reasoning on real-world images covering a broad subset of first-order logic. By categorizing samples into positive variants, near-boundary (NB), and far-from-boundary (FB) negatives, COCOLogic-V2 enables fine-grained diagnosis of model accountability. Our evaluations show that models tend to separate positive and FB samples well but fail on NB samples, while perceptual noise and large rule-induced search spaces pose additional challenges in few-shot settings. Together, these results highlight that visual inductive reasoning remains an open challenge and COCOLogic-V2 provides a concrete foundation for advancing methods in this direction.

Problem

• visual inductive reasoning
• logical inconsistency
• object-centric dataset
• first-order logic
• model accountability

Innovation

• visual inductive reasoning
• hard negatives
• object-centric dataset
• first-order logic
• model accountability