IoT-enabled multimodal large language model-based framework for compositional food recognition, nutrient profiling, NOVA classification, and cancer risk mitigation

Automated dietary assessment increasingly struggles to evaluate industrial food processing levels, revealing a systemic gap between the visual presentation of meals and their underlying ultra-processed food composition. Although existing dietary monitoring tools can effectively track caloric and macronutrient intake, they lack the qualitative processing assessments required to identify hidden additives and map dietary risk triage quadrants.

This study introduces a modular, IoT-enabled multimodal dietary decision-support framework that decouples visual food recognition, standardized database retrieval, and qualitative language reasoning for automated NOVA processing classification and 2D decision matrix mapping. Operating on isolated single-dish meal instances, the system employs a four-phase cascaded pipeline that integrates vision-language models, database retrieval, and large language models (LLMs) with strict JSON schema enforcement. The platform was evaluated using the Nutrition-NOVA validation benchmark, a harmonized multimodal dataset constructed through dense vector-space semantic linkage between quantitative profiles from the Center for Food Safety database and qualitative NOVA tiers from Open Food Facts. This benchmark addresses a 75.0% metadata sparsity deficit identified in legacy repositories.

In experimental evaluations on the Nutrition-NOVA benchmark, the baseline Qwen-VL-Max architecture achieved an adjacent accuracy of 77.7%. Furthermore, enforcing schema modularity enabled complete plug-and-play backend replaceability between commercial cloud application programming interfaces (APIs) and private, on-premises open-weight models without requiring architectural retraining.

The proposed framework establishes an efficient, safety-first screening tool for digital healthcare decision support. By isolating qualitative LLM reasoning from visual perceptual noise and database metadata sparsity, the system successfully bridges the gap between intuitive meal perception and point-of-care dietary risk triage, offering a non-invasive and scalable solution for automated dietary auditing and longitudinal population health monitoring.