Improving the Robustness of Odour Recognition with Odour-Image Data Fusion in Open-Air Settings

Odour recognition with low-cost gas sensors is challenging in open-air settings due to the non-specificity of the sensors and environmental variability. This can be mitigated by incorporating additional information into the classification process. This paper investigates odour-image multimodality in two case-studies of increasing complexity: banana ripening in open-air environment and strawberry ripening in a glasshouse environment. Data were collected using custom acquisition platforms equipped with cameras and MOX gas sensors operated with temperature modulation. For the visual modality, image classification (MobileNetV3) and object detection (YoloV5) models are trained. For the odour modality, established classical machine learning methods (Random Forest, XGBoost, SVM and Logistic Regression) and neural networks (1D-CNN, LSTM, MLP, and ELM) are employed. Each modality is analysed independently and together to critically assess scenarios in which combining modalities provides a clear advantage over using either modality alone. Results show that models trained on odour data achieve high accuracy in controlled environments but underperform in more dynamic open-air settings. Image-based models are sensitive to the image quality in all environments; however, they are more robust when deployed in different environments. Lastly, it is demonstrated that decision fusion consistently increases the accuracy, by as much as +12.36% in the banana ripening and +3.63% in the strawberry ripening scenario. Where decision fusion does not improve classification accuracy significantly, it is shown that the multimodal approach can still be leveraged to identify high-confidence predictions by selecting samples where both modalities agree on the label.