SwordBench: Evaluating Orthogonality of Steering Image Representations

2026-05-10

AI Summary

Existing evaluation protocols struggle to systematically measure how visual models’ representations change under conceptual interventions. To address this gap, this work introduces SwordBench, a novel benchmark that incorporates two new evaluation dimensions—cross-concept robustness and collateral damage—to systematically assess the efficacy of various vision backbones in concept-removal tasks. The framework employs linear SVMs, sparse autoencoders, and other methods to uniformly evaluate representational orthogonality and stability. The study reveals that while linear SVMs excel in separability and orthogonality, they fail to achieve zero collateral damage. Moreover, under standard settings, prevailing methods consistently fall short of ideal intervention performance, highlighting fundamental limitations in current approaches to interpretable and safe AI.

Abstract

Steering or intervening on model representations at inference time to correct predictions is essential for AI interpretability and safety, yet existing evaluation protocols are limited to ambiguous language modeling tasks. To address this gap, we introduce SwordBench, a benchmark for steering image representations of vision models across multiple backbones and concept removal tasks. Beyond a unified benchmarking suite, we propose new evaluation notions that uncover the second-order effects of orthogonalization among concept activation vectors for pragmatic steering. Specifically, cross-concept robustness measures the stability of concept detection performance across inputs orthogonalized against alternative concepts, and collateral damage quantifies whether steering inadvertently affects model performance on a downstream task for inputs lacking the bias. We find that although a linear support vector machine exhibits superior separability and orthogonality, it fails to achieve zero collateral damage, often trailing sparse autoencoders. In simpler regimes, both standard baselines and optimization-based methods fail to achieve perfect steering. The source code will be made available soon on GitHub.

Problem

• steering
• orthogonality
• image representations
• concept removal
• collateral damage

Innovation

• steering
• orthogonality
• concept removal
• cross-concept robustness
• collateral damage