Collaborate sim and real: Robot Bin Packing Learning in Real-world and Physical Engine

2025-11-25

AI Summary

Existing 3D bin-packing research predominantly models the task as a discrete, static optimization problem, neglecting gravity-driven continuous physical interactions—leading to poor stability and frequent collapses in real-world deployment. To bridge the reality gap, this paper proposes a simulation-to-reality co-adaptive hybrid reinforcement learning framework. It integrates domain randomization within a physics-based simulator to account for uncertainties in friction coefficients, elasticity, and mass distributions, and further refines the policy via closed-loop feedback from real robotic platforms. This synergistic approach significantly enhances the generalizability and robustness of learned policies under dynamic real-world conditions. Experiments demonstrate consistently lower collapse rates on both simulated and physical robot platforms. In large-scale logistics deployments, our method reduces collapse rate by 35% compared to baseline approaches.

Technology Category

• Intelligent Robots: Learning & Optimization for ROB
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

The 3D bin packing problem, with its diverse industrial applications, has garnered significant research attention in recent years. Existing approaches typically model it as a discrete and static process, while real-world applications involve continuous gravity-driven interactions. This idealized simplification leads to infeasible deployments (e.g., unstable packing) in practice. Simulations with physical engine offer an opportunity to emulate continuous gravity effects, enabling the training of reinforcement learning (RL) agents to address such limitations and improve packing stability. However, a simulation-to-reality gap persists due to dynamic variations in physical properties of real-world objects, such as various friction coefficients, elasticity, and non-uniform weight distributions. To bridge this gap, we propose a hybrid RL framework that collaborates with physical simulation with real-world data feedback. Firstly, domain randomization is applied during simulation to expose agents to a spectrum of physical parameters, enhancing their generalization capability. Secondly, the RL agent is fine-tuned with real-world deployment feedback, further reducing collapse rates. Extensive experiments demonstrate that our method achieves lower collapse rates in both simulated and real-world scenarios. Large-scale deployments in logistics systems validate the practical effectiveness, with a 35% reduction in packing collapse compared to baseline methods.

Problem

• Addressing simulation-to-reality gap in robot bin packing
• Improving packing stability under continuous gravity interactions
• Reducing collapse rates with hybrid RL and real feedback

Innovation

• Hybrid RL framework combines simulation and real feedback
• Domain randomization enhances generalization in physical simulation
• Fine-tuning with real-world data reduces packing collapse rates