Enhancing spectral analysis of ghee adulteration via a deep learning-based multimodal attention mechanism

Expensive dairy products like ghee are at high risk of hazardous adulteration with cheaper fats, and traditional testing approaches such as FTIR device and conventional machine learning are too slow, complex, and lack the interpretability to be useful for practical food safety. To address this, we propose a new deep learning framework which is robust and highly interpretable. Our solution is a multimodal attention-based architecture that integrates optimized machine learning classifiers with advanced deep models, including these phases Phase I: Feature Extraction by help of Machine Learning Models (e.g. SVM, XGBoost, Random Forest), Phase II: Extracted Feature converted to spectroscopy, Phase III: an Attention-Augmented 1D-CNN and a Transformer-based network. Importantly, we introduced a Spectral Attention Map (SAM) mechanism that automatically focuses the chemically relevant FTIR bands, ensuring the completely transparent model's high-accuracy decisions beyond the "black-box". The reliability can be enhanced using a Dynamic Multimodal Fusion scheme that learns to assigns weights to the outputs of all models adaptively. Our system was evaluated on a statistically enriched, custom FTIR dataset with various different attributes of ghee on the basis of which we create the spectroscopy of the respective attributes and achieved a staggering classification accuracy of 99.6% with a 94% feature dimensionality reduction, showing the efficiency and effectiveness of the proposed approach. This work establishes a new benchmark for food authentication using AI, offering a highly accurate, scalable and transparent solution with significant potential to be incorporated into a real-time and portable food safety systems to allow on-site verification.