Robust Transfer Learning with Side Information

2026-03-09

AI Summary

This work addresses the challenge of excessive conservatism in conventional distributionally robust optimization (DRO) when environments undergo significant shifts, which often leads to overly conservative policies and poor transferability. To mitigate this, the authors propose a novel approach that integrates a small number of samples from the target domain with auxiliary dynamic information—such as moment bounds, distributional distances, and density ratios—between source and target domains. By constraining the transition kernel estimation, the method constructs a tight, estimation-centered uncertainty set that preserves robustness while substantially reducing policy suboptimality and improving sample efficiency. Theoretical analysis bridges DRO, constrained kernel estimation, and finite-sample guarantees. Empirical evaluations on OpenAI Gym and classical control tasks consistently outperform both robust and non-robust transfer learning baselines.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Intelligent Robots: Learning & Optimization for ROB
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Robust Markov Decision Processes (MDPs) address environmental shift through distributionally robust optimization (DRO) by finding an optimal worst-case policy within an uncertainty set of transition kernels. However, standard DRO approaches require enlarging the uncertainty set under large shifts, which leads to overly conservative and pessimistic policies. In this paper, we propose a framework for transfer under environment shift that derives a robust target-domain policy via estimate-centered uncertainty sets, constructed through constrained estimation that integrates limited target samples with side information about the source-target dynamics. The side information includes bounds on feature moments, distributional distances, and density ratios, yielding improved kernel estimates and tighter uncertainty sets. The side information includes bounds on feature moments, distributional distances, and density ratios, yielding improved kernel estimates and tighter uncertainty sets. Error bounds and convergence results are established for both robust and non-robust value functions. Moreover, we provide a finite-sample guarantee on the learned robust policy and analyze the robust sub-optimality gap. Under mild low-dimensional structure on the transition model, the side information reduces this gap and improves sample efficiency. We assess the performance of our approach across OpenAI Gym environments and classic control problems, consistently demonstrating superior target-domain performance over state-of-the-art robust and non-robust baselines.

Problem

• Robust Transfer Learning
• Environment Shift
• Distributionally Robust Optimization
• Uncertainty Set
• Markov Decision Processes

Innovation

• Robust Transfer Learning
• Side Information
• Estimate-Centered Uncertainty Sets
• Distributionally Robust Optimization
• Markov Decision Processes