Fine-grained 3D rice phenotyping via multi-scale NeRF and multimodal segmentation.

Area of Science:

• Agricultural Science
• Computer Vision
• Data Science

Background:

• Fine-grained 3D phenotypic analysis is crucial for rice breeding and yield estimation.
• Existing 3D reconstruction methods like Neural Radiance Fields (NeRF) struggle with rice due to data volume and viewpoint sensitivity.
• Challenges in rice point cloud segmentation include occlusion and grain similarity, hindering trait extraction.

Purpose of the Study:

• To develop a comprehensive pipeline for rice data acquisition and segmentation.
• To address the limitations of existing methods in reconstructing rice point clouds under low-quality data conditions.
• To create a benchmark dataset and advanced models for rice plant analysis.

Main Methods:

• Proposed Multi-Scale NeRF (MSNeRF) with a structure-detail collaborative reconstruction and dynamic initialization density scheduling.
• Introduced a multimodal and multitask rice dataset (MMR) for benchmarking.
• Developed Vision Rice Knowledge Graph Network (VRKGNet) for point cloud segmentation, integrating image segmentation priors.

Main Results:

• MSNeRF achieved high-fidelity point cloud reconstruction using as few as 10 viewpoints.
• VRKGNet demonstrated superior rice plant segmentation, achieving 88.79% mIoU for semantic segmentation and 84.55% AP25 for instance segmentation.
• The proposed methods outperformed mainstream algorithms in reconstruction and segmentation tasks.

Conclusions:

• The developed MSNeRF and VRKGNet provide effective solutions for 3D rice reconstruction and segmentation.
• The MMR dataset serves as a valuable resource for future research in computational agriculture.
• These advancements contribute to improved rice breeding and yield estimation through detailed phenotypic analysis.