Metric Learning Encoding Models: A Multivariate Framework for Interpreting Neural Representations

2024-02-18

AI Summary

Neural representational interpretability remains a core challenge at the intersection of neuroscience and AI. This paper formalizes the mapping from theoretical features to neural activity as an **explicit metric learning problem**, introducing the first learnable metric embedding framework grounded in second-order isomorphism—jointly encoding individual features and their higher-order interactions. Our method integrates representational similarity analysis (RSA) with parametric distance metric optimization, enabling application across multimodal neural data (language, vision, audition) and arbitrary artificial or empirical neural networks. Compared to univariate approaches such as FR-RSA, our framework achieves significantly improved noise robustness and ground-truth recovery accuracy: it more precisely quantifies feature importance on synthetic benchmarks and demonstrates strong generalization on real-world language tasks (e.g., gender and tense identification). The implementation is publicly available, facilitating cross-disciplinary neural representation analysis.

Technology Category

• Machine Learning: Representation Learning
• Computer Vision: Representation Learning for Vision
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Understanding how explicit theoretical features are encoded in opaque neural systems is a central challenge now common to neuroscience and AI. We introduce Metric Learning Encoding Models (MLEMs) to address this challenge most directly as a metric learning problem: we fit the distance in the space of theoretical features to match the distance in neural space. Our framework improves on univariate encoding and decoding methods by building on second-order isomorphism methods, such as Representational Similarity Analysis, and extends them by learning a metric that efficiently models feature as well as interactions between them. The effectiveness of MLEM is validated through two sets of simulations. First, MLEMs recover ground-truth importance features in synthetic datasets better than state-of-the-art methods, such as Feature Reweighted RSA (FR-RSA). Second, we deploy MLEMs on real language data, where they show stronger robustness to noise in calculating the importance of linguistic features (gender, tense, etc.). MLEMs are applicable to any domains where theoretical features can be identified, such as language, vision, audition, etc. We release optimized code applicable to measure feature importance in the representations of any artificial neural networks or empirical neural data at https://github.com/LouisJalouzot/MLEM.

Problem

• Interpreting neural representations of theoretical features in opaque systems
• Improving multivariate analysis of neural feature interactions and distances
• Validating metric learning framework robustness on synthetic and real data

Innovation

• Metric Learning Encoding Models fit feature distances to neural distances
• MLEMs improve univariate methods by modeling feature interactions
• Framework learns metric for robust feature importance in neural data