Balancing Multiple Objectives in Urban Traffic Control with Reinforcement Learning from AI Feedback

2026-02-24

AI Summary

This work proposes a novel extension of reinforcement learning from AI feedback (RLAIF) to multi-objective adaptive traffic signal control, addressing the common limitation of existing approaches that prioritize a single dominant objective due to conflicting goals and fail to accommodate diverse user preferences. By leveraging large language models to generate multi-objective preference labels—without requiring manual annotation or intricate reward function engineering—the method trains adaptive control policies that reflect varying user priorities. Experimental results demonstrate that the proposed approach effectively balances competing objectives such as traffic efficiency, fairness, and energy consumption, significantly enhancing both the practicality and scalability of traffic signal control strategies in real-world urban environments.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Machine Learning: Learning Preferences or Rankings
• Multiagent Systems: Multiagent Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Reward design has been one of the central challenges for real world reinforcement learning (RL) deployment, especially in settings with multiple objectives. Preference-based RL offers an appealing alternative by learning from human preferences over pairs of behavioural outcomes. More recently, RL from AI feedback (RLAIF) has demonstrated that large language models (LLMs) can generate preference labels at scale, mitigating the reliance on human annotators. However, existing RLAIF work typically focuses only on single-objective tasks, leaving the open question of how RLAIF handles systems that involve multiple objectives. In such systems trade-offs among conflicting objectives are difficult to specify, and policies risk collapsing into optimizing for a dominant goal. In this paper, we explore the extension of the RLAIF paradigm to multi-objective self-adaptive systems. We show that multi-objective RLAIF can produce policies that yield balanced trade-offs reflecting different user priorities without laborious reward engineering. We argue that integrating RLAIF into multi-objective RL offers a scalable path toward user-aligned policy learning in domains with inherently conflicting objectives.

Problem

• multi-objective reinforcement learning
• RL from AI feedback
• urban traffic control
• preference-based learning
• conflicting objectives

Innovation

• Multi-objective Reinforcement Learning
• Reinforcement Learning from AI Feedback (RLAIF)
• Large Language Models
• Preference-based Learning
• Urban Traffic Control