Towards Learning Boulder Excavation with Hydraulic Excavators

2025-09-22

AI Summary

Autonomous removal of large, irregular rocks on construction sites using standard excavator buckets faces significant challenges—including sparse LiDAR perception, dust interference, occlusions, dynamically varying soil resistance, and unstructured environments. Method: We propose the first end-to-end reinforcement learning–based autonomous excavation method that requires no specialized grippers; it operates solely on per-rock 20-point LiDAR inputs and proprioceptive sensing. Training leverages high-fidelity simulation integrating rigid-body dynamics and analytical soil mechanics models to enable soil-adaptive, closed-loop control for both dragging and penetration behaviors. Contribution/Results: On a 12-ton physical excavator, our method achieves a 70% rock removal success rate for 0.4–0.7 m rocks—comparable to human operators’ 83%—demonstrating, for the first time, the feasibility of learning complex discrete-object manipulation with standard equipment under harsh outdoor conditions.

Technology Category

• Intelligent Robots: Manipulation
• Search and Optimization: Learning to Search
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Construction sites frequently require removing large rocks before excavation or grading can proceed. Human operators typically extract these boulders using only standard digging buckets, avoiding time-consuming tool changes to specialized grippers. This task demands manipulating irregular objects with unknown geometries in harsh outdoor environments where dust, variable lighting, and occlusions hinder perception. The excavator must adapt to varying soil resistance--dragging along hard-packed surfaces or penetrating soft ground--while coordinating multiple hydraulic joints to secure rocks using a shovel. Current autonomous excavation focuses on continuous media (soil, gravel) or uses specialized grippers with detailed geometric planning for discrete objects. These approaches either cannot handle large irregular rocks or require impractical tool changes that interrupt workflow. We train a reinforcement learning policy in simulation using rigid-body dynamics and analytical soil models. The policy processes sparse LiDAR points (just 20 per rock) from vision-based segmentation and proprioceptive feedback to control standard excavator buckets. The learned agent discovers different strategies based on soil resistance: dragging along the surface in hard soil and penetrating directly in soft conditions. Field tests on a 12-ton excavator achieved 70% success across varied rocks (0.4-0.7m) and soil types, compared to 83% for human operators. This demonstrates that standard construction equipment can learn complex manipulation despite sparse perception and challenging outdoor conditions.

Problem

• Autonomously removing large irregular boulders using standard excavator buckets
• Handling unknown rock geometries in harsh outdoor environments with poor perception
• Adapting excavation strategies to varying soil resistance without specialized tools

Innovation

• Reinforcement learning policy trained in simulation
• Processes sparse LiDAR points and proprioceptive feedback
• Controls standard excavator buckets for rock extraction