DIAL: Distribution-Informed Adaptive Learning of Multi-Task Constraints for Safety-Critical Systems

2025-01-30

AI Summary

Safety-critical systems (e.g., autonomous driving) suffer from poor generalization and reliance on handcrafted constraint functions in multi-task safe reinforcement learning. Method: We propose the first distribution-aware multi-task constraint adaptation framework, which models cross-task constraint distributions to enable constraint knowledge transfer and dynamically calibrates risk sensitivity via expert demonstrations to mitigate safety generalization bottlenecks induced by expert bias. Technically, it integrates imitation learning, implicit constraint distribution modeling, risk-aware optimization, and multi-task meta-learning. Contribution/Results: Evaluated on control and navigation tasks, our method achieves significant improvements in both safety and task success rate—without requiring any task-specific constraint definitions—outperforming all existing baselines comprehensively.

Technology Category

• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Multiagent Systems: Multiagent Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Safe reinforcement learning has traditionally relied on predefined constraint functions to ensure safety in complex real-world tasks, such as autonomous driving. However, defining these functions accurately for varied tasks is a persistent challenge. Recent research highlights the potential of leveraging pre-acquired task-agnostic knowledge to enhance both safety and sample efficiency in related tasks. Building on this insight, we propose a novel method to learn shared constraint distributions across multiple tasks. Our approach identifies the shared constraints through imitation learning and then adapts to new tasks by adjusting risk levels within these learned distributions. This adaptability addresses variations in risk sensitivity stemming from expert-specific biases, ensuring consistent adherence to general safety principles even with imperfect demonstrations. Our method can be applied to control and navigation domains, including multi-task and meta-task scenarios, accommodating constraints such as maintaining safe distances or adhering to speed limits. Experimental results validate the efficacy of our approach, demonstrating superior safety performance and success rates compared to baselines, all without requiring task-specific constraint definitions. These findings underscore the versatility and practicality of our method across a wide range of real-world tasks.

Problem

• Adaptive Learning
• Multi-task Rules
• Autonomous Driving Safety

Innovation

• Adaptive Learning
• Risk Assessment
• Multi-task Safety Rules