Fine-tuning Aligned Classifiers for Merging Outputs: Towards a Superior Evaluation Protocol in Model Merging

2024-12-18

AI Summary

In model merging, representational misalignment—modeled as an orthogonal transformation—exists between fused outputs and fine-tuned classifiers in the feature space, leading to evaluation distortion and suboptimal performance. To address this, we propose a novel few-shot unsupervised classifier alignment paradigm: using only a small number of unlabeled samples, it calibrates classifier weights via orthogonal transformation to achieve feature-space alignment. Based on this, we establish a more reliable evaluation protocol for merged models. Experiments across multiple classification tasks demonstrate that our method significantly improves the accuracy of merged models and yields evaluations that more faithfully reflect the intrinsic capabilities of merging methods. This work introduces a new benchmark for model merging that jointly ensures effectiveness and evaluability.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Multi-modal Vision
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Model merging combines multiple fine-tuned models into a single one via parameter fusion, achieving improvements across many tasks. However, in the classification task, we find a misalignment issue between merging outputs and the fine-tuned classifier, which limits its effectiveness. In this paper, we first demonstrate the following observations: (1) Merging outputs exhibit the comparable cluster effect with fine-tuned outputs, and already contain necessary classification information; (2) The misalignment between merging outputs and the fine-tuned classifier can converge to an orthogonal transformation, and alleviating this misalignment can significantly enhance the performance of merging models. Based on these observations, we then propose a new protocol FT-Classifier, which fine-tunes an aligned classifier with few-shot unlabeled samples, enabling better evaluation of merging methods and improved classification performance.

Problem

• Address misalignment in model merging outputs
• Enhance classification via orthogonal transformation
• Propose FT-Classifier for superior evaluation

Innovation

• Fine-tunes aligned classifiers
• Uses few-shot unlabeled samples
• Improves model merging performance