SSMNBench: Diagnosing Image-based Cross-View Human-Object Understanding via Single-View Sufficiency and Multi-View Necessity

2026-06-24

AI Summary

This work addresses the lack of effective evaluation of cross-view reasoning capabilities in current vision-language models within complex multi-view human-object interaction scenarios, where existing benchmarks fail to disentangle robustness to visual distractions from genuine cross-view evidence integration. To this end, we introduce SSMNBench—a novel diagnostic benchmark comprising 3,300 question-answer pairs—and propose a principled task framework centered on Single-View Sufficiency (SVS) and Multi-View Necessity (MVN) to systematically assess model reasoning under varying viewpoint conditions. Through geometric and semantic viewpoint perturbation experiments across 17 state-of-the-art multimodal large models, we reveal a prevalent reliance on single-image semantic averaging and viewpoint bias rather than true cross-view fusion. Notably, models exhibit significant “interference degradation” under redundant views, failing to coherently integrate fragmented geometric evidence—providing critical diagnostic insights for future cross-view architecture design.

Abstract

Multimodal Large Language Models (MLLMs) have shown remarkable progress in single-image perception, yet their ability to reason about complex cross-view human-centric scenes remains largely unverified. Current multi-view benchmarks evaluate models using a fixed "bag of frames" and thus conflate a model's robustness to visual distraction with its genuine ability to fuse fragmented cross-view evidence. To address this issue, we introduce SSMNBench, a diagnostic benchmark comprising 3,300 curated QA pairs for cross-view human and human-object understanding. SSMNBench uniquely categorizes tasks into Single-View Sufficiency (SVS) and Multi-View Necessity (MVN). By systematically perturbing view availability across 17 state-of-the-art MLLMs, critical limitations are revealed: models suffer from severe "distraction degradation" when presented with redundant views (SVS), and fail to integrate fragmented geometric evidence across cameras (MVN). Our evaluations demonstrate that modern MLLMs rely on multiple single-image semantic averaging and view preference rather than genuine cross-view synthesis. By exposing these fundamental vulnerabilities, SSMNBench provides a rigorous diagnostic framework to drive the advancement of future cross-view-aware multimodal architectures. The code is available at: $ \href{https://github.com/gtc-gh/SSMNBench}{\text{SSMNBench}} $

Problem

• cross-view understanding
• multimodal large language models
• visual distraction
• evidence fusion
• human-object interaction

Innovation

• SSMNBench
• Cross-View Understanding
• Single-View Sufficiency
• Multi-View Necessity
• Multimodal Large Language Models