TeaNeRF: an integrated 3D visual perception pipeline for tea bud harvesting.

Area of Science:

• Computer Vision
• Robotics
• Agricultural Technology

Background:

• Accurate tea bud perception is crucial for intelligent tea harvesting, but faces challenges like small size, occlusion, and complex backgrounds.
• Existing methods struggle with precise 3D spatial information required for harvesting planning in natural tea plantations.

Purpose of the Study:

• To develop an integrated 3D visual perception pipeline, TeaNeRF, for accurate harvesting-oriented tea bud analysis.
• To address limitations in detecting, segmenting, and spatially analyzing tea buds in complex plantation settings.

Main Methods:

• TeaNeRF integrates sequential 2D recognition (YOLO), prompt-guided segmentation, and monocular depth estimation with Neural Radiance Fields (NeRF) for 3D reconstruction.
• The pipeline utilizes depth supervision and semantic-aware neural fields to generate dense, geometrically consistent semantic point clouds.
• A 3D clustering and geometric fitting strategy is employed for tea bud counting and harvesting candidate point estimation.

Main Results:

• TeaNeRF demonstrated significant improvements in detection accuracy (mAP@50 = 91.7%) and segmentation quality (IoU = 0.640) on a dataset of 4,700 images.
• The system achieved enhanced reconstruction fidelity, evidenced by increased PSNR and reduced LPIPS.
• 3D semantic point cloud analysis enabled feasible tea bud counting and provided spatial guidance for harvesting planning.

Conclusions:

• TeaNeRF offers a robust solution for 3D tea bud perception, overcoming challenges in natural environments.
• The pipeline provides practical, structured 3D spatial information (locations, counts, candidate points) for automated tea harvesting systems.
• This approach facilitates precise harvesting planning by delivering reliable perception-level outputs.