Neuron-Aware Data Selection for Annotation-Free LLM Self-Distillation

2026-07-02

AI Summary

This work proposes an unsupervised self-distillation framework for domain adaptation of large language models in the absence of human annotations or authentic interaction feedback. The approach uniquely leverages internal neuron activation signals to guide both training data selection and teacher context construction, integrating online policy distillation to learn from the teacher’s output distribution. By combining pseudo-label generation, context augmentation, and activation-driven data filtering, the framework substantially outperforms existing annotation-free self-evolution methods across multiple specialized domain benchmarks. Notably, it not only enhances in-domain performance but also effectively preserves cross-domain generalization and model calibration.

Abstract

Post-training large language models (LLMs) without real-world interaction feedback or human-labeled supervision remains challenging, particularly in specialized domains where expert annotations are costly to obtain. Recent annotation-free self-evolution methods address this by using the model's own outputs as supervision signals, constructing a teacher via additional context and aggregating predictions across multiple rollouts through majority voting to produce pseudo-labels. However, these approaches are not without drawbacks: SFT- and GRPO-based variants suffer out-of-domain performance degradation, while reward-based on-policy RL inflates calibration error. In this paper, we propose Neuron On-Policy Self-Distillation (Neuron-OPSD), a data-centric framework for annotation-free self-distillation that leverages internal neuron activations to guide both training-data selection and teacher context construction. The model is then trained via on-policy distillation from the teacher distribution, requiring no ground-truth labels at any stage. Across specialized-domain benchmarks, Neuron-OPSD improves in-domain task performance while preserving cross-domain generalization and mitigating calibration collapse over prior annotation-free baselines. This framework is particularly relevant to settings where online interaction or external supervision is costly or infeasible, and is conceptually distinct from offline RL approaches that rely on logged, reward-labeled trajectories.

Problem

• annotation-free
• self-distillation
• large language models
• post-training
• data selection

Innovation

• neuron activation
• annotation-free
• self-distillation
• on-policy learning
• data selection