OPA-Pack: Object-Property-Aware Robotic Bin Packing

2025-05-19

AI Summary

Existing robotic bin-packing systems focus solely on geometric constraints, neglecting critical semantic attributes—such as fragility, edibility, and chemical compatibility—leading to safety hazards and material incompatibilities in real-world deployment. This work addresses e-commerce and warehouse scenarios by introducing, for the first time, multi-dimensional semantic attributes into autonomous packing planning. We propose an attribute-aware packing framework featuring a joint optimization mechanism for incompatible-pair separation and pressure mitigation on fragile items. We construct the first large-scale dataset with semantic attribute annotations for 1,032 everyday objects. To enable fine-grained attribute recognition, we integrate retrieval-augmented generation with chain-of-thought reasoning, and develop OPA-Net—a novel architecture incorporating attribute embeddings and dual height maps (for fragility and avoidance)—alongside a custom deep Q-network decision module. Experiments show a 43-percentage-point improvement in incompatible-pair separation accuracy (from 52% to 95%), a 29.4% reduction in compressive stress on fragile objects, and optimal packing compactness, validated on a physical robot platform.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Robotic bin packing aids in a wide range of real-world scenarios such as e-commerce and warehouses. Yet, existing works focus mainly on considering the shape of objects to optimize packing compactness and neglect object properties such as fragility, edibility, and chemistry that humans typically consider when packing objects. This paper presents OPA-Pack (Object-Property-Aware Packing framework), the first framework that equips the robot with object property considerations in planning the object packing. Technical-wise, we develop a novel object property recognition scheme with retrieval-augmented generation and chain-of-thought reasoning, and build a dataset with object property annotations for 1,032 everyday objects. Also, we formulate OPA-Net, aiming to jointly separate incompatible object pairs and reduce pressure on fragile objects, while compacting the packing. Further, OPA-Net consists of a property embedding layer to encode the property of candidate objects to be packed, together with a fragility heightmap and an avoidance heightmap to keep track of the packed objects. Then, we design a reward function and adopt a deep Q-learning scheme to train OPA-Net. Experimental results manifest that OPA-Pack greatly improves the accuracy of separating incompatible object pairs (from 52% to 95%) and largely reduces pressure on fragile objects (by 29.4%), while maintaining good packing compactness. Besides, we demonstrate the effectiveness of OPA-Pack on a real packing platform, showcasing its practicality in real-world scenarios.

Problem

• Enables robots to consider object properties like fragility and edibility during packing
• Develops a framework to separate incompatible objects and reduce pressure on fragile items
• Maintains packing compactness while improving property-aware packing accuracy

Innovation

• Retrieval-augmented generation for property recognition
• OPA-Net for joint incompatible separation and fragility reduction
• Deep Q-learning with property embedding and heightmaps