Visual-Based Forklift Learning System Enabling Zero-Shot Sim2Real Without Real-World Data

2024-12-16

arXiv.org

AI Summary

Automated counterbalanced forklifts face challenges due to the absence of a safe, verifiable development environment and heavy reliance on real-world data. Method: This paper proposes a zero-shot Sim2Real paradigm: a high-fidelity rasterized digital twin is constructed from CAD models; an end-to-end vision-to-action mapping network is designed and trained via pure simulation-driven deep reinforcement learning using the Proximal Policy Optimization (PPO) algorithm—eliminating heuristic tuning and real-world data collection entirely. Contribution/Results: The policy trained solely in simulation achieves direct transfer to the physical system. On a 1:14-scale experimental platform, the method attains a 60% success rate on pallet loading tasks—constituting the first empirical validation of forklift autonomous operation trained without any real-world data. This work establishes a novel, safe, and efficient pathway toward automation for industrial mobile robots.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Planning, Routing, and Scheduling: Learning for Planning and Scheduling

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Forklifts are used extensively in various industrial settings and are in high demand for automation. In particular, counterbalance forklifts are highly versatile and employed in diverse scenarios. However, efforts to automate these processes are lacking, primarily owing to the absence of a safe and performance-verifiable development environment. This study proposes a learning system that combines a photorealistic digital learning environment with a 1/14-scale robotic forklift environment to address this challenge. Inspired by the training-based learning approach adopted by forklift operators, we employ an end-to-end vision-based deep reinforcement learning approach. The learning is conducted in a digitalized environment created from CAD data, making it safe and eliminating the need for real-world data. In addition, we safely validate the method in a physical setting utilizing a 1/14-scale robotic forklift with a configuration similar to that of a real forklift. We achieved a 60% success rate in pallet loading tasks in real experiments using a robotic forklift. Our approach demonstrates zero-shot sim2real with a simple method that does not require heuristic additions. This learning-based approach is considered a first step towards the automation of counterbalance forklifts.

Problem

• Automating counterbalance forklifts without real-world data
• Lacking safe, verifiable development environment for forklift automation
• Achieving zero-shot sim2real transfer for forklift tasks

Innovation

• Photorealistic digital learning environment for safe training
• End-to-end vision-based deep reinforcement learning approach
• Zero-shot sim2real without real-world data