Model Correlation Detection via Random Selection Probing

2025-09-28

AI Summary

This paper addresses the lack of reliable criteria for detecting fine-tuning relationships or identity between large language models (LLMs) and vision-language models (VLMs). We propose Randomized Selection Probing (RSP), a statistical hypothesis testing framework that elicits transfer behavior via prefix tuning, integrates random task sampling, and calibrates against irrelevant model baselines to produce interpretable, principled p-values for transparent provenance attribution. Its core contribution is the first unified, empirically verifiable paradigm for model relatedness detection—operational even when model parameters are inaccessible. Experiments demonstrate that RSP robustly distinguishes related from unrelated models across both text and vision modalities (yielding significantly smaller vs. larger p-values), maintains stability across diverse architectures and training configurations, and substantially outperforms existing heuristic similarity-based methods.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The growing prevalence of large language models (LLMs) and vision-language models (VLMs) has heightened the need for reliable techniques to determine whether a model has been fine-tuned from or is even identical to another. Existing similarity-based methods often require access to model parameters or produce heuristic scores without principled thresholds, limiting their applicability. We introduce Random Selection Probing (RSP), a hypothesis-testing framework that formulates model correlation detection as a statistical test. RSP optimizes textual or visual prefixes on a reference model for a random selection task and evaluates their transferability to a target model, producing rigorous p-values that quantify evidence of correlation. To mitigate false positives, RSP incorporates an unrelated baseline model to filter out generic, transferable features. We evaluate RSP across both LLMs and VLMs under diverse access conditions for reference models and test models. Experiments on fine-tuned and open-source models show that RSP consistently yields small p-values for related models while maintaining high p-values for unrelated ones. Extensive ablation studies further demonstrate the robustness of RSP. These results establish RSP as the first principled and general statistical framework for model correlation detection, enabling transparent and interpretable decisions in modern machine learning ecosystems.

Problem

• Detecting model correlation without parameter access
• Providing statistical evidence via hypothesis testing framework
• Distinguishing fine-tuned models from unrelated ones rigorously

Innovation

• Statistical hypothesis testing for model correlation detection
• Optimizing transferable prefixes for random selection tasks
• Unrelated baseline model filtering to reduce false positives