A Multi-Head Attention Transformer Model for Wearable in Situ Fall Detection

Falls represent a critical health hazard for elderly individuals, often leading to severe injuries and decreased quality of life. While existing fall detection systems predominantly rely on threshold-based algorithms or conventional machine learning approaches like CNNs and RNNs, they frequently exhibit limitations in capturing the complex temporal dependencies and spatial features inherent in fall data. This paper presents a novel multi-head attention transformer architecture specifically designed for wearable in situ fall detection. Our approach leverages the transformer's self-attention mechanism to effectively model long-range dependencies in inertial measurement unit (IMU) sensor data collected from a shinbone-mounted device. The proposed architecture introduces several key innovations: 1) a position-aware embedding layer that preserves the temporal structure of IMU signals, 2) multi-head self-attention blocks that simultaneously attend to different signal features across varying time scales, and 3) a hybrid feature fusion module that combines global context with local patterns. Experimental results on our human subjects dataset demonstrate the critical importance of addressing dataset imbalance. A baseline model trained on imbalanced data achieved only 48.0% fall sensitivity, rendering it unsafe for practical use. By training our proposed transformer architecture on a large, balanced dataset with a specialized Focal Loss function, we increased fall sensitivity to 90.5% on the test set while maintaining an overall accuracy of 93.1% (validation best epoch: sensitivity 92.5% and specificity 95.8%, see Fig. 3). Furthermore, we present a complete hardware/software co-design, including a custom-designed wearable PCB with optimized power management and a comprehensive strategy for deploying our model on an ultra-low-power Lattice Semiconductor iCE40UP5K FPGA, achieving efficient operation with only 22 mA current consumption. We frame this study as technical feasibility and edge deployment validation; direct elderly clinical validation remains future work.