NearID: Identity Representation Learning via Near-identity Distractors

2026-04-02

AI Summary

Existing visual encoders are prone to interference from background context in identity-related tasks, leading to unreliable representations. To address this, this work introduces the NearID dataset and a Strict Similarity-based Recognition (SSR) evaluation protocol, which isolates identity as the sole discriminative signal by fixing backgrounds and introducing semantically similar but identity-different distractors to eliminate contextual shortcuts. The authors propose a two-stage contrastive learning objective optimized on frozen backbones to refine identity representations, establishing the first evaluation framework centered on near-identity interference. Their approach dramatically improves identity discrimination accuracy from 30.7% to 99.2%, enhances part-level discriminability by 28.0%, and significantly increases alignment with human judgments on DreamBench++.

Abstract

When evaluating identity-focused tasks such as personalized generation and image editing, existing vision encoders entangle object identity with background context, leading to unreliable representations and metrics. We introduce the first principled framework to address this vulnerability using Near-identity (NearID) distractors, where semantically similar but distinct instances are placed on the exact same background as a reference image, eliminating contextual shortcuts and isolating identity as the sole discriminative signal. Based on this principle, we present the NearID dataset (19K identities, 316K matched-context distractors) together with a strict margin-based evaluation protocol. Under this setting, pre-trained encoders perform poorly, achieving Sample Success Rates (SSR), a strict margin-based identity discrimination metric, as low as 30.7% and often ranking distractors above true cross-view matches. We address this by learning identity-aware representations on a frozen backbone using a two-tier contrastive objective enforcing the hierarchy: same identity > NearID distractor > random negative. This improves SSR to 99.2%, enhances part-level discrimination by 28.0%, and yields stronger alignment with human judgments on DreamBench++, a human-aligned benchmark for personalization. Project page: https://gorluxor.github.io/NearID/

Problem

• identity representation
• vision encoders
• contextual entanglement
• personalized generation
• image editing

Innovation

• Near-identity Distractors
• Identity Representation Learning
• Contrastive Learning
• Context Disentanglement
• Personalized Generation