KBF: Knowledge Boundary as Fingerprint for Language Model and Black-Box API Auditing

2026-05-28

AI Summary

This work addresses the lack of authenticity verification mechanisms in existing intermediary or resold large language model (LLM) APIs. The authors propose the KBF protocol, which leverages the stability of numerical recall near a model’s knowledge boundary as a unique fingerprint. Through black-box auditing, boundary probing, and statistical hypothesis testing, KBF enables low-cost, highly robust detection of API authenticity. The method effectively identifies economically motivated model substitutions and platform-model inconsistencies, correctly flagging all 155 instances of such substitutions across 16 production-grade LLM endpoints without false rejections. Furthermore, it reveals significant discrepancies between reference endpoints and services on 7 out of 27 evaluated platforms, particularly affecting high-tier Claude offerings.

Abstract

Relay and reseller APIs increasingly intermediate access to large language models (LLMs), but users have no direct way to verify that a claimed endpoint is actually serving the advertised model. We introduce KBF, a low-cost black-box auditing protocol that fingerprints model APIs using stable numerical recall near the knowledge boundary. Across 16 production LLM endpoints, KBF flags all 155 economically relevant substitutions without rejecting any same-model controls, remains stable under deployment variation, detects high-separation mixed-routing attacks when only 5-10% of traffic is substituted, and finds that 7 of 27 platform model cells in a six-platform shadow API audit are statistically inconsistent with their reference endpoints, with inconsistencies concentrated on premium Claude endpoints.

Problem

• Language Model
• Black-Box API
• Model Substitution
• API Auditing
• Knowledge Boundary

Innovation

• Knowledge Boundary
• Model Fingerprinting
• Black-Box Auditing
• LLM API Verification
• Mixed-Routing Attack Detection