Adaptive RGB-VNIR spectral fusion for non-destructive quality assessment of Canadian wild rice kernels

Canadian wild rice kernels exhibit diverse postharvest variations, including breakage, discoloration, insect damage, and incomplete dehulling, making accurate and non-destructive classification important for screening and grading. However, RGB imaging may be insufficient for distinguishing kernels with subtle or overlapping visual characteristics, while visible-near-infrared (VNIR) spectra provide complementary information but contain substantial redundancies. Moreover, conventional multimodal fusion strategies based on direct concatenation or fixed weighting cannot readily adapt to sample-specific differences in modality contribution.

To address these limitations, this study proposed a lightweight RGB-VNIR dual-modal fusion framework for fine-grained quality-category classification of Canadian wild rice kernels. A Coordinate Attention-enhanced MobileNetV3-Small was employed to extract spatial phenotypic features, while a 1D-GhostResNet-SE was used to model continuous VNIR spectral responses. A Cross-Modal Contribution-Aware Recalibration Fusion Head (CMCR-FH) was further introduced to adaptively regulate the contributions of the two modalities. The framework was evaluated on 2,048 paired RGB-VNIR samples from eight categories using stratified five-fold cross-validation.

The proposed framework achieved a mean accuracy of 98.68% ± 0.22% and a Macro-F1 score of 98.69% ± 0.22%, representing numerical improvements of 4.93 and 4.97 percentage points, respectively, over the baseline model, while requiring only 1.236 M parameters and 63.537 M multiply-accumulate operations. Ablation experiments demonstrated complementary contributions from visual feature enhancement, lightweight spectral representation, and sample-adaptive multimodal fusion, with CMCR-FH producing the largest numerical performance gain among the evaluated components. Grad-CAM++ and SHAP analyses further indicated that the model jointly utilized visible phenotypic abnormalities and class-dependent spectral response patterns for discrimination.

These results demonstrate that integrating complementary RGB and VNIR information with sample-adaptive fusion can improve fine-grained quality-category classification while maintaining low computational complexity. The proposed framework provides an effective balance among classification accuracy, computational efficiency, and interpretability, offering a promising approach for multimodal postharvest quality-category classification and grading of Canadian wild rice kernels.