Establishing Trustworthy LLM Evaluation via Shortcut Neuron Analysis

2025-06-04

AI Summary

Existing LLM evaluations suffer severely from contamination of public benchmark data, leading to inflated and inaccurate capability estimates. Method: Instead of constructing new dynamic benchmarks, this work introduces the novel concept of “shortcut neurons”—neurons that develop biased activation patterns during training—and proposes a causal contrastive analysis to identify them. Building on this, we establish the first “shortcut neuron repair” evaluation paradigm, which applies neuron-level causal interventions, parameter sensitivity analysis, and dynamic neural repair—without modifying any benchmark. Contribution/Results: Experiments across multiple benchmarks and hyperparameter configurations demonstrate that our method significantly mitigates contamination effects. Evaluation scores align closely with those of the trusted, uncontaminated MixEval benchmark (Spearman ρ > 0.95), substantially improving the accuracy and reliability of LLM capability assessment.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The development of large language models (LLMs) depends on trustworthy evaluation. However, most current evaluations rely on public benchmarks, which are prone to data contamination issues that significantly compromise fairness. Previous researches have focused on constructing dynamic benchmarks to address contamination. However, continuously building new benchmarks is costly and cyclical. In this work, we aim to tackle contamination by analyzing the mechanisms of contaminated models themselves. Through our experiments, we discover that the overestimation of contaminated models is likely due to parameters acquiring shortcut solutions in training. We further propose a novel method for identifying shortcut neurons through comparative and causal analysis. Building on this, we introduce an evaluation method called shortcut neuron patching to suppress shortcut neurons. Experiments validate the effectiveness of our approach in mitigating contamination. Additionally, our evaluation results exhibit a strong linear correlation with MixEval, a recently released trustworthy benchmark, achieving a Spearman coefficient ($ ho$) exceeding 0.95. This high correlation indicates that our method closely reveals true capabilities of the models and is trustworthy. We conduct further experiments to demonstrate the generalizability of our method across various benchmarks and hyperparameter settings. Code: https://github.com/GaryStack/Trustworthy-Evaluation

Problem

• Addressing data contamination in LLM evaluation benchmarks
• Identifying shortcut neurons causing model overestimation
• Proposing trustworthy evaluation via shortcut neuron patching

Innovation

• Identifies shortcut neurons via comparative analysis
• Proposes shortcut neuron patching evaluation method
• Validates method with high correlation benchmarks