The Turning Point of 3D Plant Phenotyping: 3D Foundation Models Enable Minute-to-Second Cross-Crop Reconstruction and Beyond

2026-07-02

AI Summary

Traditional 3D plant phenotyping struggles with sparse viewpoints and self-occlusion in low-cost imagery—such as smartphone videos—resulting in inefficient and inaccurate reconstructions. This work proposes the first cross-crop phenotyping framework based on 3D foundation models (3DFMs), establishing an end-to-end pipeline that transforms sparse inputs into organ-level semantic segmentation and metrically accurate 3D reconstructions. The approach integrates feedforward geometry recovery, geometry-constrained 3D Gaussian splatting, iterative view synthesis, 2D-to-3D semantic transfer, and scale recovery. Evaluated on 26 plant sequences, the method reduces average reconstruction time from 6.52 minutes to 1.58 seconds while preserving high-fidelity geometry and precise phenotypic measurements, substantially enhancing throughput and scalability.

Abstract

3D plant phenotyping is notoriously known to be procedure-complicated and of low throughput due to the extensive multi-view imaging, the fragile 3D reconstruction pipeline, and the additional cost from reconstructed geometry to phenotypic extraction. These limitations are further amplified in low-cost data acquisition, where smartphone videos or sparsely sampled multi-view images provide limited view overlap and self-occlusion. In this work, we show that the conventional 3D plant phenotyping pipeline could be streamlined and significantly accelerated with 3D Foundation Models (3DFMs), and particularly, present one of the first cross-crop 3D phenotyping frameworks powered by 3DFMs. The framework replaces COLMAP-style sparse initialization with 3DFM-based feed-forward geometric recovery, combines geometry-constrained 3D Gaussian Splatting for dense reconstruction, enables few-view reconstruction through iterative view synthesis and refinement, and converts reconstructed geometry into measurable organs through 2D-to-3D semantic transfer, metric scale recovery, and organ instance separation. We further construct a cross-crop dataset with smartphone-based image acquisition, diverse plant morphologies, and manual annotations for segmentation and phenotypic evaluation. Experiments across 26 plant sequences show that 3D Foundation Models reduce the average reconstruction time from 6.52 minutes to 1.58 seconds while maintaining high reconstruction quality and phenotyping accuracy. These results suggest a fresh technical route for high-throughput 3D plant phenotyping, from low-cost image acquisition to fast reconstruction, perception, scale recovery, and phenotypic measurement.

Problem

• 3D plant phenotyping
• low-cost data acquisition
• sparse multi-view imaging
• self-occlusion
• throughput limitation

Innovation

• 3D Foundation Models
• 3D Gaussian Splatting
• cross-crop phenotyping
• few-view reconstruction
• semantic 2D-to-3D transfer