Using Convolutional Neural Networks with Multiple Thermal Sensors for Unobtrusive Pose Recognition

Area of Science:

• Computer Vision
• Artificial Intelligence
• Biomedical Engineering

Background:

• Enabling individuals needing 24/7 care to live at home requires understanding their actions.
• Recognizing Activities of Daily Living (ADLs) is key, but privacy concerns often limit monitoring solutions.
• This research prioritizes unobtrusive pose recognition to maintain inhabitant privacy.

Purpose of the Study:

• To develop a privacy-preserving pose recognition system for monitoring Activities of Daily Living (ADLs).
• To investigate the effectiveness of Convolutional Neural Networks (CNNs) with thermal imagery for pose estimation.
• To evaluate different sensor configurations for optimal performance, cost, and privacy.

Main Methods:

• Implemented a novel Convolutional Neural Network (CNN) architecture, inspired by AlexNet.
• Utilized thermopile infrared sensors (TISs) to capture thermal imagery in a smart kitchen environment.
• Trained and evaluated the CNN model using an original dataset of thermal images.

Main Results:

• Achieved a high F1-score of 0.9920 using five ceiling-mounted TISs.
• Investigated limitations of ceiling-based TIS and evaluated corner-based configurations.
• Demonstrated promising F1-scores (0.9266, 0.9149, 0.8468) with two to four corner TISs, balancing performance and cost.

Conclusions:

• The developed CNN-based pose recognition system effectively estimates poses from thermal imagery.
• The system offers a viable, privacy-preserving solution for monitoring Activities of Daily Living (ADLs) in home environments.
• Sensor configuration significantly impacts performance, cost, and privacy, with corner-based setups offering flexibility.