SAGE: Style-Adaptive Generalization for Privacy-Constrained Semantic Segmentation Across Domains

2025-12-02

AI Summary

To address the degradation of semantic segmentation model generalization under privacy-constrained settings due to domain shift, this paper proposes SAGE—a parameter-free, weight-agnostic input-level domain adaptation method. SAGE generates diverse source-domain representations via style transfer and jointly optimizes feature distribution alignment with dynamic style-aware prompt fusion, thereby constructing transferable visual prompts directly in the input space to enable efficient adaptation of frozen models to unseen domains. Extensive experiments across five benchmark datasets demonstrate that SAGE significantly outperforms existing parameter-free adaptation approaches under strict privacy constraints, while also surpassing full fine-tuning baselines. Notably, SAGE is the first method to achieve synergistic optimization of high-fidelity style guidance and lightweight prompt fusion—enabling effective, privacy-preserving domain adaptation without accessing or modifying model parameters.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Security and Privacy: Data transparency and provenance

Abstract

Domain generalization for semantic segmentation aims to mitigate the degradation in model performance caused by domain shifts. However, in many real-world scenarios, we are unable to access the model parameters and architectural details due to privacy concerns and security constraints. Traditional fine-tuning or adaptation is hindered, leading to the demand for input-level strategies that can enhance generalization without modifying model weights. To this end, we propose a extbf{S}tyle- extbf{A}daptive extbf{GE}neralization framework ( extbf{SAGE}), which improves the generalization of frozen models under privacy constraints. SAGE learns to synthesize visual prompts that implicitly align feature distributions across styles instead of directly fine-tuning the backbone. Specifically, we first utilize style transfer to construct a diverse style representation of the source domain, thereby learning a set of style characteristics that can cover a wide range of visual features. Then, the model adaptively fuses these style cues according to the visual context of each input, forming a dynamic prompt that harmonizes the image appearance without touching the interior of the model. Through this closed-loop design, SAGE effectively bridges the gap between frozen model invariance and the diversity of unseen domains. Extensive experiments on five benchmark datasets demonstrate that SAGE achieves competitive or superior performance compared to state-of-the-art methods under privacy constraints and outperforms full fine-tuning baselines in all settings.

Problem

• Improves generalization of frozen models under privacy constraints
• Synthesizes visual prompts to align feature distributions across styles
• Adapts to unseen domains without modifying model weights or architecture

Innovation

• Synthesizes visual prompts for feature alignment
• Adaptively fuses style cues based on input context
• Enhances generalization without modifying model weights