Synthesizing and Reconstructing Missing Sensory Modalities in Behavioral Context Recognition.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Human activity and context detection are crucial for applications like assisted living.
• Multimodal data used for context prediction are often imbalanced, noisy, and contain missing values.
• Real-world scenarios frequently involve missing sensor data, hindering context inference.

Purpose of the Study:

• To propose a novel method for handling missing sensory features in multimodal data.
• To synthesize realistic samples for robust context prediction models.
• To evaluate the performance of the proposed method against classical approaches.

Main Methods:

• Utilized an adversarial autoencoder for feature imputation and data synthesis.
• Developed a fully-connected classification network by extending an encoder.
• Evaluated multi-label classification performance with missing modalities on a large-scale dataset.

Main Results:

• The proposed method demonstrated superior performance in filling missing values compared to classical approaches.
• Systematic evaluation showed robust multi-label classification performance despite missing modalities.
• Class-conditional data generation exhibited strong generative power for context classification.

Conclusions:

• Adversarial autoencoders are effective for handling missing sensory data in human activity recognition.
• The proposed method enables reliable context inference even with incomplete multimodal sensor information.
• The approach shows significant potential for real-world applications requiring robust activity and context detection.