reinforcement learning

This study investigates effective coordination mechanisms for cooperation in multi-agent large language model (LLM) systems, focusing on coordination problems and social dilemmas. We comparatively evaluate two paradigms—direct communication and curriculum learning—across two canonical game-theoretic settings: the 4-player Stag Hunt and the iterated public goods game with punishment. Results show that introducing a minimal “cheap talk” channel—requiring only single-character utterances—increases cooperation rates from 0% to 48.3%, demonstrating the robustness and efficiency of lightweight communication as a coordination mechanism. In contrast, structured curriculum learning leads to a 27.4% average payoff reduction among agents and induces learned pessimism. The work innovatively reveals the low-threshold, high-return nature of communication for multi-agent cooperation, while cautioning against unintended negative effects of curriculum design. These findings provide scalable, interpretable intervention strategies for enhancing collaborative behavior in LLM-based multi-agent systems.

Existing GUI reinforcement learning approaches face two key challenges: (1) neglecting task difficulty heterogeneity, leading to poor training adaptability, and (2) relying on coarse-grained reward signals, resulting in inefficient policy updates. To address these, we propose a curriculum-based fine-grained optimization framework comprising three core components: (1) a trajectory difficulty grouping mechanism for adaptive task difficulty ranking; (2) a multi-signal reward function integrating rule-based priors and model-driven judgments to enhance feedback precision; and (3) Group Relative Policy Optimization (GRPO), a novel algorithm enabling dynamic curriculum adjustment and stable policy optimization. Evaluated on the Android Control public benchmark, our method achieves a 5.6% absolute improvement over prior state-of-the-art methods; on an internal online benchmark, it yields a 10.3% gain in success rate. Overall, the framework significantly boosts success rates across diverse GUI navigation tasks.

Training deep reinforcement learning agents under multi-objective conflicting rewards often suffers from instability and difficulty balancing task performance against constraint satisfaction. To address this, we propose a two-stage reward curriculum learning framework: an initial phase optimizes a simplified reward to accelerate convergence, followed by a smooth transition to the full, complex reward. We introduce a novel Actor-Critic fidelity criterion for automatic, dynamic stage switching and design a flexible replay buffer enabling cross-phase sample reuse. Our approach integrates curriculum learning, dynamic reward shaping, and adaptive experience replay. Evaluated on the DeepMind Control Suite—including tasks with explicit constraints—and real-world mobile robot navigation, our method significantly outperforms non-curriculum baselines, achieving a more robust trade-off between task success rate and constraint violation rate.

In multi-agent reinforcement learning (MAS) under sparse rewards, training inefficiency and policy fragility arise from delayed feedback and difficulty in sharing experience across agents. To address these challenges, this paper proposes a collaborative curriculum learning framework. Its key contributions are: (1) a multidimensional curriculum design jointly modulating task difficulty, agent count, and environmental complexity; (2) a variational evolutionary algorithm for automated subtask generation; and (3) a co-evolutionary mechanism integrating agent policy optimization with environmental model learning. The framework unifies curriculum learning, variational evolution, MAS, and environment modeling. Evaluated on five cooperative benchmarks—including MPE and Hide-and-Seek—our method achieves significant improvements over state-of-the-art approaches: 2.1× faster convergence on average and an 18.7% increase in success rate, demonstrating both effectiveness and generalizability.

Current reinforcement learning (RL) frameworks lack native, non-intrusive support for curriculum learning (CL), requiring invasive code modifications to implement. To address this, we propose CLib—the first lightweight, general-purpose curriculum learning library. CLib features a unified API and modular architecture comprising: (i) an environment-agnostic curriculum scheduler, (ii) a distributed sampling adapter, and (iii) a cross-framework bridging layer supporting both PyTorch and TensorFlow backends. It integrates seamlessly with five+ mainstream RL libraries—including Ray RLlib and CleanRL—without altering underlying training logic. We demonstrate the first successful application of CL in complex environments NetHack and Neural MMO, and validate CLib across nine benchmark tasks, consistently outperforming state-of-the-art baselines. By eliminating implementation barriers, CLib lowers the entry threshold for CL adoption, promotes standardization, and enhances reproducibility in RL research.

Current large language models are constrained by fixed context windows, limiting their ability to handle highly complex tasks. This work proposes a reinforcement learning–based recursive agent training framework that enables agents, during inference, to autonomously decide whether and how to recursively invoke themselves, dynamically decomposing tasks and delegating subtasks. The approach achieves, for the first time, adaptive recursion and coordination among agents at inference time, effectively circumventing context length limitations. It significantly enhances generalization and reasoning efficiency on tasks far exceeding the complexity encountered during training, while maintaining higher training efficiency and achieving lower overall inference latency compared to single-agent systems.

This work addresses the challenge of designing a global reward function in heterogeneous multi-agent reinforcement learning that effectively accommodates diverse agent objectives. To this end, the authors propose MAGPIE, a method that leverages expert preference signals to construct individual reward models for each agent without requiring a predefined global reward. A monotonic aggregation mechanism is introduced to combine these agent-specific rewards into a unified global objective for policy optimization. Theoretically, this approach uniquely bridges agent-specific preference modeling with Nash equilibrium optimization, proving that decentralized preference learning converges to a Nash equilibrium. Empirical results demonstrate that MAGPIE achieves performance on par with handcrafted reward functions across standard multi-agent benchmarks and sequential production-line scenarios, validating its effectiveness in eliminating the need for precise reward engineering.

This study addresses the problem of dynamically allocating prediction tasks among capacity-constrained agents—whether human or artificial—to maximize collective performance. It introduces, for the first time, a theoretical formulation of task assignment under explicit capacity constraints and proposes a context-aware sequential exploration–exploitation learning framework. This framework integrates multi-agent capability modeling with optimized task–agent matching strategies. Empirical evaluations demonstrate that the proposed approach significantly outperforms non-contextual baselines across tabular, image, and text prediction tasks, and is effective in collaborative settings involving both large language models and human agents.

This work addresses the vulnerability of cooperative resilience—the capacity of a multi-agent system to anticipate, resist, recover from, and adapt to disturbances—in mixed-motive environments to reward design. It proposes a novel framework that explicitly optimizes for cooperative resilience as a learning objective by inferring reward functions from ranked behavioral trajectories, leveraging preference-based reward learning with three parameterization strategies: linear models, handcrafted features, and neural networks. By integrating a hybrid reward scheme that combines individual task rewards with resilience-inferred rewards, the approach significantly enhances system robustness in social dilemma settings. Empirical results demonstrate that this resilience-oriented reward design maintains task performance while substantially reducing the risk of systemic collapse and resource overexploitation, thereby underscoring its critical role in fostering sustainable cooperation.

Traditional reinforcement learning is constrained by predefined rewards and closed environments, limiting its capacity for autonomous goal setting and long-term planning in open-ended scenarios. This work proposes integrating large language models (LLMs) into reinforcement learning frameworks to endow agents with cognitive-like capabilities—such as metareasoning, introspection, and multi-step decision-making—thereby enabling goal generation, dynamic policy adaptation, and interactive reasoning. By transcending the limitations of static objectives and episodic interactions, the approach establishes a theoretical foundation and design paradigm for LLM-driven cognitive agents. The study systematically identifies key challenges and outlines promising directions for future research, advancing reinforcement learning toward a cognitive agent paradigm.