Private Anytime Selective-Risk Certification for Federated Retrieval-Augmented Generation: Guarantees and Empirical Limits

2026-08-08

AI Summary

This work addresses the challenge of providing provable error-rate guarantees for accepted outputs in retrieval-augmented generation (RAG) systems operating under federated learning, differential privacy, and dynamic monitoring constraints. The authors propose Fed-SRC, the first method integrating differential privacy, federated learning, and anytime selective risk control. By constructing a martingale-based e-process from Gaussian-perturbed scores and loss histograms, Fed-SRC enables dynamic client participation and adaptive thresholding while preserving privacy. Risk is uniformly controlled via indexed logging and noise variance, and a total variation term facilitates calibration-to-deployment distribution shift adaptation. Experiments demonstrate that Fed-SRC consistently satisfies simultaneous confidence bounds across all settings: on the HaluEval QA task, it achieves 100% certification success at target rejection rate \( r^* = 0.20 \) in non-private scenarios, whereas naive private baselines fail 73%–99% of the time; even under strong privacy (\( \varepsilon \leq 4 \)), it remains effective with only about 30× more streaming events than calibration samples.

Abstract

Selective-risk certificates promise that accepted outputs meet a declared error target. We develop Fed-SRC, a score-agnostic certificate for federated, differentially private, adaptively monitored retrieval-augmented generation. Clients release only Gaussian-perturbed score and loss histograms. Record-indexed and noise-variance-indexed martingales jointly bound target-risk contrast and accepted mass over all registered thresholds and rounds, permitting predictable recruitment, dropout, threshold selection, and optional stopping. A range-one total-variation term transfers the calibration mixture to a declared deployment mixture. The contribution is this private, federated, anytime combination, rather than the contrast statistic or acceptance floor individually. Empirically, no simultaneous-bound violation occurs in any evaluated cell, privacy level, or policy. Operational power depends on the score and population: the primary target r*=0.10 never certifies, and on RAGTruth the secondary target r*=0.20 never certifies either, whereas on HaluEval question answering it certifies in all 200 non-private trials, with held-out risk below the target. Naively privatized non-private certificates violate their bounds in 146 to 198 of 200 trials. As an exploratory comparison, we also evaluate a private betting-capital heuristic for which we do not establish e-process validity. This heuristic stops certifying at epsilon <= 4, where Fed-SRC still certifies. Certification nevertheless consumes roughly 30 times more stream events than unique calibration items.

Problem

• selective-risk certification
• federated learning
• differential privacy
• retrieval-augmented generation
• anytime guarantees

Innovation

• Selective-risk certification
• Federated learning
• Differential privacy
• Retrieval-augmented generation
• Anytime guarantees