Driver Head-Hand Cooperative Action Recognition Based on FMCW Millimeter-Wave Radar and Deep Learning

Driver status plays a critical role in ensuring driving safety. However, the current visual recognition-based methods for detecting driver actions and status are often limited to factors such as ambient light condition, occlusion, and privacy concerns. In contrast, millimeter-wave radar offers various advantages such as high accuracy, ease of integration, insensitivity to light condition, and low cost; therefore, it has been widely used for monitoring vital signals and in action recognition. Despite this, the existing studies on driver action recognition have been hindered by limited accuracy and a narrow range of detectable actions. In this study, we utilized a 77 GHz millimeter-wave frequency-modulated continuous-wave radar to construct a dataset encompassing seven types of driver head-hand cooperative actions. Furthermore, a deep learning network model based on VGG16-LSTM-CBAM using micro-Doppler spectrograms as input was developed for action classification. The experimental results demonstrated that, compared to the existing CNN-LSTM and ALEXNET-LSTM networks, the proposed network achieves a classification accuracy of 99.16%, effectively improving driver action detection.