Certified Robustness under Heterogeneous Perturbations via Hybrid Randomized Smoothing

2026-05-12

AI Summary

Existing randomized smoothing methods support only a single modality and struggle to provide robustness guarantees for multimodal models under joint perturbations of discrete (e.g., text) and continuous (e.g., image) inputs. This work proposes a unified randomized smoothing framework that factorizes discrete and continuous noise distributions and constructs a Neyman–Pearson hypothesis test under the worst-case joint perturbation, thereby delivering the first model-agnostic, analytically tractable robustness certificate for heterogeneous multimodal inputs. The approach rigorously generalizes both Gaussian and discrete randomized smoothing. Empirical validation on multimodal safety filtering tasks demonstrates its effectiveness, offering the first provably robust guarantee against joint text-and-image perturbations.

Abstract

Randomized smoothing provides strong, model-agnostic robustness certificates, but existing guarantees are limited to single modalities, treating continuous and discrete inputs in isolation. This limitation becomes critical in multimodal models, where decisions depend on cross-modal semantics and adversaries can jointly perturb heterogeneous inputs, rendering unimodal certificates insufficient. We introduce a unified randomized smoothing framework for mixed discrete--continuous inputs based on an analytically tractable Neyman--Pearson formulation of the joint worst-case problem. By analyzing the joint likelihood ordering induced by factorized discrete and continuous noise, our approach yields a closed-form, one-dimensional certificate that strictly generalizes both Gaussian (image-only) and discrete (text-only) randomized smoothing. We validate the framework on multimodal safety filtering, providing, to our knowledge, the first model-agnostic Neyman--Pearson certificate for joint discrete-token and continuous-image perturbations in interaction-dependent text--image safety filtering.

Problem

• Certified Robustness
• Heterogeneous Perturbations
• Multimodal Models
• Randomized Smoothing
• Discrete-Continuous Inputs

Innovation

• randomized smoothing
• heterogeneous perturbations
• multimodal robustness
• Neyman-Pearson certification
• discrete-continuous inputs