diversity maintenance

This study investigates how behavioral diversity influences team performance in multi-agent reinforcement learning (MARL), particularly under computational constraints and sparse reward settings, to enhance collaboration efficiency and robustness. We propose a trajectory-embedding-based behavioral distance metric, a diversity-regularized objective, heterogeneous policy initialization, and a curriculum-style perturbation training framework. Our work is the first to systematically demonstrate that behavioral heterogeneity spontaneously induces unbiased role specialization, strengthens morphological synergy, accelerates cooperative policy discovery under sparse rewards, and enables implicit skill retention and transfer. Experiments across diverse collaborative tasks show that heterogeneous teams achieve 23–41% higher average task success rates than homogeneous baselines, recover from environmental perturbations 2.8× faster, and more stably acquire reusable collaborative sub-policies.

Unsupervised skill discovery faces a fundamental trade-off between behavioral quality and diversity—particularly challenging in high-dimensional robotic control domains with rich latent skill spaces. To address this, we propose Contrastive Multi-objective Skill Discovery (ComSD), the first framework to jointly optimize a contrastive learning–driven diversity reward and a particle-filter–based exploration reward, thereby establishing a dynamic multi-objective reward mechanism in a reward-free setting. ComSD integrates adaptive weight balancing and unsupervised representation learning to unify high-fidelity skill discrimination with progressive discovery of novel behaviors. Evaluated on 32 downstream tasks, ComSD achieves state-of-the-art performance, significantly enhancing both behavioral diversity and practical utility of multi-joint robots across hierarchical exploration regimes.

Existing multi-agent reinforcement learning (MARL) methods predominantly focus on individual policy optimization while neglecting inter-agent interaction, leading to policy homogenization and poor adaptability. To address this, we propose CoDiCon—a novel MARL framework that, for the first time, integrates the sociological concept of constructive conflict into MARL. CoDiCon introduces a centralized intrinsic reward mechanism grounded in rank-based features, implemented via a parameterized reward module and a bilevel optimization architecture to dynamically balance competition and cooperation: competitive intrinsic rewards stimulate policy exchange and enhance diversity, while centralized rank-based features ensure collaborative consistency. Evaluated on the SMAC and Google Research Football (GRF) benchmarks, CoDiCon significantly outperforms state-of-the-art methods, demonstrating synergistic improvements in policy diversity, team adaptability, and task performance.

This work addresses the collapse of generation diversity in reinforcement fine-tuning, where optimization dynamics often drive model outputs toward a single solution (i.e., a Dirac delta distribution) due to misalignment between the objective function and the optimization landscape. To mitigate this, we propose DRIFT, the first framework to systematically incorporate diversity incentives into reinforcement fine-tuning. DRIFT synergistically preserves both task alignment and output diversity during policy updates through reward-concentrated subset sampling, stochastic prompt augmentation, and potential-based reward shaping. Experimental results demonstrate that DRIFT achieves Pareto superiority: it improves generation diversity by 9.08%–43.46% while maintaining equivalent task alignment, or enhances task alignment by 59.65%–65.86% under comparable diversity levels.

To address policy homogenization induced by parameter sharing in multi-agent reinforcement learning—particularly its inability to accommodate heterogeneous agent identities and task requirements—this paper proposes a zero-overhead, identity-driven adaptive subnet partitioning mechanism. Inspired by neural functional parcellation, the method employs a learnable identity encoder to generate agent-specific binary masks, which dynamically route inputs to localized subnetworks within a shared backbone, thereby enabling differentiated policy representations. Crucially, it introduces no additional parameters and is fully compatible with standard on-policy algorithms such as PPO and A2C. Extensive experiments on StarCraft II, the Multi-Agent Particle Environment (MPE), and custom heterogeneous benchmarks demonstrate an average 12.7% improvement in win rate and a 3.2× increase in inter-agent policy diversity, significantly outperforming both conventional parameter sharing and Hypernetwork-based baselines.

Traditional reinforcement learning relies on deterministic policies, which struggle to meet the demand for behavioral diversity in tasks such as language model fine-tuning or scientific discovery. This work proposes a novel paradigm based on distributions over reward functions, introducing nonlinear objectives over action sets and leveraging a principled gradient estimator derived from contextual bandits. The approach enables controllable induction of policy diversity without compromising expected return. By unifying classical policy gradient methods with action-set optimization frameworks, the proposed method demonstrates robust generation of diverse behaviors in complex tasks, significantly outperforming conventional approaches in empirical evaluations.

This work addresses the issue of premature convergence in multi-agent multi-objective optimization, which often arises from behavioral homogenization. To mitigate this, the study introduces a behavioral entropy maximization mechanism into multi-objective evolutionary algorithms for the first time. Specifically, within the NSGA-II framework, it integrates policy entropy rewards with multi-objective fitness evaluation to explicitly promote behavioral diversity while preserving Pareto optimality. This approach effectively alleviates behavioral collapse and substantially enhances exploration capability. Experimental results in the rover domain demonstrate that, compared to the NSGA-II baseline, the proposed method achieves up to a 48% improvement in hypervolume metric, along with significantly enhanced solution set quality and diversity.

This work addresses the persistent stagnation often encountered by multimodal embodied agents in long-horizon tasks due to overreliance on a single planning strategy, which hinders recovery from failure. The study introduces quality-diversity (QD) optimization into embodied planning—proposing a framework that offline constructs a diverse repertoire of strategies and online adaptively switches among them. Diversity is fostered through experience-guided recombination and mutation, while a behavior-space index is built upon interaction intensity and goal-directedness. During execution, the system continuously monitors task progress and, upon detecting stagnation, rolls back and switches to a behaviorally distinct alternative strategy. Evaluated on the ThreeDWorld transportation benchmark, the approach significantly improves task success rates and interaction efficiency, demonstrating that strategic diversity is crucial for adaptive planning and robust failure recovery.

This work addresses the limitations of single-policy exploration in large-scale reinforcement learning and the instability or inefficiency often caused by existing ensemble methods due to excessive exploration. Through theoretical analysis, the study reveals the critical role of policy diversity in learning efficiency and proposes a coupled policy optimization approach. By introducing a KL divergence constraint within an ensemble policy gradient framework, the method explicitly regulates inter-policy diversity to enable efficient and stable structured exploration. Evaluated in large-scale parallel reinforcement learning systems, the proposed approach significantly outperforms baseline methods—including SAPG, PBT, and PPO—on complex tasks such as dexterous manipulation, achieving substantial improvements in both sample efficiency and final performance.

This study addresses a key limitation in existing reinforcement learning approaches, which often overlook individual differences when modeling biological behavior, focusing instead on optimal policies or population averages. To overcome this constraint, the work introduces a biologically interpretable framework that integrates methods from multiple subfields of reinforcement learning to construct a computational model capable of generating diverse individual behaviors. By systematically synthesizing technical strategies that support behavioral diversity, the research establishes a novel paradigm for modeling individual variation in biological agents. This paradigm effectively narrows the gap between simulated and real-world biological behaviors, offering both a theoretical foundation and practical guidance for future research in biologically plausible behavior modeling.