Intelligent evaluation of black tea withering quality via image-spectrum fusion using a multimodal attention network...

Withering is a critical process that significantly impacts the final quality of black tea. Traditional assessment methods, which rely on manual evaluation, are often subjective and time-consuming. To overcome this limitation, we proposed a multimodal evaluation approach based on computer vision (CV) and near-infrared spectroscopy (NIRs), integrating both qualitative classification and quantitative prediction during the black tea withering process. For withering degree classification, we developed a multimodal attention fusion network that combines a CV feature extraction module (pyramid CNN), an NIRs feature extraction module (attention-enhanced CNN and gated recurrent unit), and a cross-modal fusion mechanism using contrastive learning, achieving an accuracy of 95.56 %. For quantitative assessment, we constructed a squeeze-and-excitation enhanced ResNet18 model to fuse CV and NIRs features for predicting moisture and soluble sugar content, yielding test set correlation coefficients of 0.9151 and 0.9023, respectively. These innovations provide a comprehensive strategy for intelligent withering evaluation in black tea withering.