iHearken: Chewing sound signal analysis based food intake recognition system using Bi-LSTM softmax network

Food ingestion is an integral part of health and wellness. Continues monitoring of different food types and observing the amount being consumed prevents gastrointestinal diseases and weight-related issues. Food intake recognition (FIR) systems, thus have significant impact on everyday life. The purpose of this study is to develop an automatic approach for the FIR using a contemporary wearable hardware and machine learning technique. This will assist clinicians and concern person to manage health issues associated with food intake.

In this work, we present a novel hardware iHearken, a headphone-like wearable sensor-based system to monitor eating activities and recognize food intake type in the free-living condition. State-of-the-art hardware is designed for data acquisition where 16 subjects are recruited and 20 different food items are used for data collection. Further, chewing sound signals are analyzed for FIR using bottleneck features. The proposed model is divided into 4 distinct phases: data acquisition, event detection using a pre-trained model, bottleneck feature extraction, and classification based on bidirectional long short-term memory (Bi-LSTM) softmax model. The Bi-LSTM network with softmax function is applied to calculate the identification score for apiece chewing signal which further classifies the chewing signal data into liquid / solid food classes.

The results of proposed model performance is evaluated in (%) for accuracy, precision, recall and F-score as 97.422, 96.808, 98.0, and 97.512, respectively, and root mean square error (RMSE), and mean absolute percentage error (MAPE) as 0.160 1.030 respectively for numbers of correct food type recognized. Further, we also evaluated our model's performance for food classification into solid and liquid and achieved an accuracy (96.66%), precision (96.40%), recall (95.230%), F-score (95.79%), RMSE (0.182), and MAPE (2.22). We also demonstrated that the food recognition accuracy of different models with the proposed model differed statistically.

An informatics complexity study of the proposed model was subsequently explored to review the effectiveness of the proposed wearable device and the methodology. The medical importance of this investigation is the reliable monitoring of the clinical development of the food intake classification methods via food chew event detection in the ambulatory environment has been justified.