🤖 AI Summary
This work addresses the challenge that existing content provenance mechanisms are ill-suited for embeddings generated by foundation models, as their identity is easily compromised by operations such as quantization, projection, or fine-tuning. The authors propose a robust watermarking scheme cryptographically bound to C2PA manifests, wherein a commitment derived from a signed locality-sensitive hash (LSH) is embedded into the C2PA sidecar file rather than recomputed during verification. The approach introduces a provably secure detection boundary centered on a single scalar parameter β. Theoretical analysis shows detection efficacy hinges on the spectral threshold eff_rank(Σ)/d ≈ 0.19. Combining block-diagonal orthogonal rotation, public whitening, input-dependent LSH, and per-vector randomness, the method enables lightweight verification—achieving AUROC ≥ 0.965 across 11 encoders, with only 284 microseconds per verification and 24 bytes of sidecar overhead per vector. The solution has been integrated into Qdrant and is compatible with the C2PA SDK.
📝 Abstract
Foundation-model embeddings are now a primary data asset, but the content-provenance machinery built for images and audio does not transfer to them. C2PA binds to an asset with a stable bit-level or perceptual identity; embeddings have neither, since quantisation, projection, fine-tuning, and windowed averaging reshape them in normal use and break any fixed hash.
We present HaloMark, a watermark for embedding vectors cryptographically bound to a C2PA manifest. It composes four standard primitives -- a block-diagonal orthogonal rotation, public whitening, an input-dependent LSH commitment, and a per-vector nonce -- around one protocol change: the producer signs the LSH commitment c into the C2PA sidecar, and the verifier reads c from the manifest instead of recomputing it. Recomputing is fragile under whitening, which flips the commitment bucket on 62% of inputs at cos = 0.96; reading the signed c reduces the verifier's score to T = T_null + beta(A)*epsilon, so security turns on a single scalar beta, which we bound rigorously for linear and non-adaptive attackers and characterise empirically for the adaptive case.
We evaluate against an adversary holding polynomially many clean/watermarked pairs under one key with full sidecar visibility, across eight baselines and ten adaptive attackers including denoising-autoencoder removal. The eleven encoders separate at an empirical threshold eff_rank(Sigma)/d ~= 0.19: above it, detection AUROC stays at 0.98 or higher across every in-budget attack on the three encoders we sweep in full, and at 0.965 or higher under single-seed DAE removal on the rest; below it every variant we tested fails. Why the threshold is dimension-uniform is left open. Deployed as a Qdrant admission filter, the verifier runs at 284 us and 24 bytes of sidecar per vector, validated end-to-end against three C2PA reference-SDK bindings.