Improving Unsupervised Task-driven Models of Ventral Visual Stream via Relative Position Predictivity

2025-05-13

AI Summary

Existing unsupervised, task-driven models predominantly employ contrastive learning to emulate the ventral visual stream (VVS) for object recognition, achieving high neural response similarity with biological vision but neglecting the VVS’s potential role in spatial representation. Method: We propose, for the first time, the cognitive hypothesis that the VVS supports relative position (RP) prediction—a fundamental spatial function—and introduce a novel joint unsupervised framework integrating RP prediction with contrastive learning via multi-task optimization. Contribution/Results: Our framework significantly enhances RP predictability without compromising object recognition performance. Empirical results demonstrate a positive correlation between RP predictability and model–brain neural representational similarity, providing critical computational evidence that the VVS subserves both object identity and spatial localization. This advances biologically plausible modeling of visual cortex by unifying functional roles previously attributed solely to dorsal-stream mechanisms.

Technology Category

• Computer Vision: Representation Learning for Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Representation Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Psychology-informed user models and recommender systems
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Based on the concept that ventral visual stream (VVS) mainly functions for object recognition, current unsupervised task-driven methods model VVS by contrastive learning, and have achieved good brain similarity. However, we believe functions of VVS extend beyond just object recognition. In this paper, we introduce an additional function involving VVS, named relative position (RP) prediction. We first theoretically explain contrastive learning may be unable to yield the model capability of RP prediction. Motivated by this, we subsequently integrate RP learning with contrastive learning, and propose a new unsupervised task-driven method to model VVS, which is more inline with biological reality. We conduct extensive experiments, demonstrating that: (i) our method significantly improves downstream performance of object recognition while enhancing RP predictivity; (ii) RP predictivity generally improves the model brain similarity. Our results provide strong evidence for the involvement of VVS in location perception (especially RP prediction) from a computational perspective.

Problem

• Enhance ventral visual stream models with relative position prediction
• Improve object recognition and brain similarity via integrated learning
• Validate VVS involvement in location perception computationally

Innovation

• Integrates relative position prediction with contrastive learning
• Enhances object recognition and brain similarity
• Models ventral visual stream more biologically realistically