Plant3R: Fusing 3D feature learning with Gaussian splatting to enhance wheat plant 3D reconstruction precision

Precise reconstruction of plant phenotypes is crucial for smart agriculture. Conventional methods struggle with low efficiency and strong dependency on high-quality data, especially for low-texture and structurally complex crops like wheat. We propose a novel 3D reconstruction framework-Plant3R-that fuses deep feature learning with 3D Gaussian Splatting (3DGS). It innovatively uses the Matching and Stereo 3D Reconstruction (MASt3R) model for sparse point cloud reconstruction and camera pose estimation via its 3D feature matching capabilities, which substantially improve image matching rates and the quality of sparse point clouds. Subsequently, 3DGS is employed for rendering and optimization, enabling end-to-end, high-fidelity, and high-robust 3D reconstruction of wheat plants. Validated on potted wheat at multiple growth stages using handheld images, our experimental results demonstrate that Plant3R performs well in feature extraction and matching, and the reconstructed point cloud provides a good geometric prior for the subsequent rendering stage. In most scenes, its key rendering metrics-Peak Signal-to-Noise Ratio (PSNR) > 34, Structural Similarity Index Measure (SSIM) of 0.94, and Learned Perceptual Image Patch Similarity (LPIPS) < 0.26-surpassed Neural Radiance Fields (NeRF) and the original 3DGS. Moreover, extracted phenotypic traits such as plant height, leaf length, and width showed high correlation with manual measurements (R² > 0.94), confirming its utility for accurate and quantitative phenotype analysis. Overall, Plant3R not only improves the rendering quality and geometric precision of 3D modeling, but also provides a reliable tool for accurate phenotypic parameter extraction and high-throughput crop phenotyping in precision agriculture.