Distributionally Robust Set Representation Learning Under Inference-Time Element Corruption

2026-05-28

AI Summary

This work addresses the performance degradation of set representation learning during inference caused by element-level corruptions, such as outliers or missing entries. To mitigate this issue, the authors propose the SW-DRSO framework, which leverages distributionally robust optimization. By introducing a differentiable barycentric adversarial mechanism, SW-DRSO efficiently transforms the intractable worst-case search over infeasible set perturbations into an optimization problem over simplex weights. This enables the model to minimize the expected loss over potentially corrupted sets during training. Experimental results across four tasks demonstrate that SW-DRSO not only maintains high overall performance but also significantly enhances robustness against element-level corruptions encountered at inference time.

Abstract

Standard Set Representation Learning methods typically excel on curated data but often overlook the challenge of inference-time element corruption. This refers to scenarios where deployed models encounter element-level degradations, such as outliers or missing components, that may distort set representation and degrade performance. We propose SW-DRSO, a distributionally robust optimization framework tailored for sets. Rather than minimizing loss solely on observed training data, SW-DRSO optimizes a tractable surrogate of the worst-case expected loss over a family of plausible inference-time variations. We introduce a barycentric adversary that approximates the intractable search over corrupted sets by a differentiable training-time optimization over simplex weights. Extensive experiments across four tasks demonstrate that SW-DRSO effectively enhances robustness against corruption while maintaining high overall performance.

Problem

• Set Representation Learning
• Inference-Time Corruption
• Distributionally Robust Optimization
• Element-Level Degradation
• Outliers

Innovation

• Distributionally Robust Optimization
• Set Representation Learning
• Inference-Time Corruption
• Barycentric Adversary
• Worst-Case Expected Loss