Reinforcement Learning Enhancement Using Vector Semantic Representation and Symbolic Reasoning for Human-Centered Autonomous Emergency Braking

2026-02-04

AI Summary

This work addresses the limitations of existing camera-based deep reinforcement learning approaches for autonomous emergency braking, which often lack high-level scene context integration and rely on rigid reward functions, thereby compromising safety and human-like decision-making. To overcome these challenges, the authors propose a neuro-symbolic reinforcement learning framework that leverages semantic segmentation to extract predicates and constructs neuro-symbolic representations of dynamic entities by incorporating spatial and shape information. Crucially, the framework introduces, for the first time, a soft first-order logic (SFOL)-based reward mechanism that embeds human values into a symbolic reasoning module, enabling context-aware and value-aligned decision-making. Evaluated in the CARLA simulation environment under diverse traffic densities and occlusion conditions, the proposed method significantly outperforms baseline approaches, demonstrating substantial improvements in both safety and policy robustness.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Machine Learning: Neuro-Symbolic Learning
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The problem with existing camera-based Deep Reinforcement Learning approaches is twofold: they rarely integrate high-level scene context into the feature representation, and they rely on rigid, fixed reward functions. To address these challenges, this paper proposes a novel pipeline that produces a neuro-symbolic feature representation that encompasses semantic, spatial, and shape information, as well as spatially boosted features of dynamic entities in the scene, with an emphasis on safety-critical road users. It also proposes a Soft First-Order Logic (SFOL) reward function that balances human values via a symbolic reasoning module. Here, semantic and spatial predicates are extracted from segmentation maps and applied to linguistic rules to obtain reward weights. Quantitative experiments in the CARLA simulation environment show that the proposed neuro-symbolic representation and SFOL reward function improved policy robustness and safety-related performance metrics compared to baseline representations and reward formulations across varying traffic densities and occlusion levels. The findings demonstrate that integrating holistic representations and soft reasoning into Reinforcement Learning can support more context-aware and value-aligned decision-making for autonomous driving.

Problem

• Reinforcement Learning
• Autonomous Emergency Braking
• Scene Context
• Reward Function
• Human-Centered AI

Innovation

• neuro-symbolic representation
• Soft First-Order Logic
• symbolic reasoning
• autonomous emergency braking
• human-centered reinforcement learning