Global Safe Sequential Learning via Efficient Knowledge Transfer

2024-02-22

arXiv.org

AI Summary

Sequence learning under safety constraints struggles to traverse disconnected safe regions, limiting exploration to local safe neighborhoods. Method: We propose the first safe transfer sequential learning framework, leveraging offline source-task data to guide safe exploration in the target task. Our approach integrates Gaussian process modeling, safe Bayesian optimization, and knowledge transfer, incorporating a safety-aware confidence upper bound constraint and an efficient, scalable precomputation-based approximation algorithm. Contribution/Results: We provide theoretical guarantees on both safety preservation and convergence. Experiments across multiple benchmark tasks demonstrate that our framework significantly reduces sample complexity, achieves— for the first time—global coverage across disconnected safe regions, and maintains computational efficiency comparable to state-of-the-art methods.

Technology Category

• Search and Optimization: Learning to Search
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs

Abstract

Sequential learning methods, such as active learning and Bayesian optimization, aim to select the most informative data for task learning. In many applications, however, data selection is constrained by unknown safety conditions, motivating the development of safe learning approaches. A promising line of safe learning methods uses Gaussian processes to model safety conditions, restricting data selection to areas with high safety confidence. However, these methods are limited to local exploration around an initial seed dataset, as safety confidence centers around observed data points. As a consequence, task exploration is slowed down and safe regions disconnected from the initial seed dataset remain unexplored. In this paper, we propose safe transfer sequential learning to accelerate task learning and to expand the explorable safe region. By leveraging abundant offline data from a related source task, our approach guides exploration in the target task more effectively. We also provide a theoretical analysis to explain why single-task method cannot cope with disconnected regions. Finally, we introduce a computationally efficient approximation of our method that reduces runtime through pre-computations. Our experiments demonstrate that this approach, compared to state-of-the-art methods, learns tasks with lower data consumption and enhances global exploration across multiple disjoint safe regions, while maintaining comparable computational efficiency.

Problem

• Safe Exploration
• Sequential Learning
• Risk Management

Innovation

• Safe Transfer Sequential Learning
• Efficient Computing Strategy
• Exploration in Distant Safe Regions