Relational Knowledge Distillation Brings DNN Representations Close Enough to Humans to Be Aligned Without Supervision

2026-08-27

AI Summary

研究使用关系知识蒸馏(RKD)方法优化深度神经网络(DNN)，使其内部表征与人类心理表征更接近，无需监督即可实现个体对象层面的对齐。

Abstract

Linking the internal representations of deep neural networks (DNNs) to human mental representations is important for using DNNs as computational models of human vision. Existing DNN representations remain insufficiently similar to human mental representations, which are not directly observable and are therefore commonly measured through large-scale similarity judgments of object images. A natural approach to narrowing this gap is to directly transfer the relational structure of human representations into DNNs, and previous studies have reported improved human-DNN representational similarity. However, whether this improvement holds under stricter evaluation remains untested in two respects: fine-grained alignment at the individual-object level, and generalization to a human embedding derived from a dataset independent of the training data. Here, we employ an unsupervised comparison method, Gromov-Wasserstein optimal transport (GWOT), which estimates human-DNN correspondences from the internal distance structure alone and thereby tests fine-grained alignment. We further assess generalization on a curated test set of concepts non-overlapping with the training data. We show that fine-tuning pre-trained DNNs with Relational Knowledge Distillation (RKD), an established relational transfer method, brings DNNs close enough to humans to be aligned at the individual-object level on this test set. We also show that this improvement is driven by a more human-like global structure, as reflected in the ordering of distances among coarse categories, while the local human-DNN nearest-neighbor overlap rate remains largely unchanged. These findings indicate that relational transfer from humans brings the global structure of pre-trained DNNs close enough to the human structure to enable fine-grained human-DNN alignment without supervision.

Problem

• Relational Knowledge Distillation
• human-DNN alignment
• unsupervised comparison
• Gromov-Wasserstein optimal transport
• fine-grained alignment

Innovation

• Relational Knowledge Distillation
• Gromov-Wasserstein optimal transport
• unsupervised alignment