NAPS Fusion: A framework to overcome experimental data limitations to predict human performance and cognitive task...

Area of Science:

• Human performance and cognitive research
• Machine learning applications
• Anomaly detection

Background:

• Experimental data in human performance research often suffers from low sample sizes, class imbalances, and conflicting ground truth labels.
• Traditional dimensionality reduction methods may fail to appropriately map data or can capture irrelevant noise.
• Integrating new sensor data requires costly remodeling of existing machine learning paradigms due to dependencies.

Purpose of the Study:

• To address uncertainty and ignorance in multi-classification machine learning problems common in human performance research.
• To develop a modular machine learning approach that accommodates new sensor integration and ambiguous ground truth data.
• To improve the accuracy and robustness of predictive models in human-centric research.

Main Methods:

• Leveraged insights from Dempster-Shafer theory (DST), stacking of machine learning models, and bagging.
• Proposed a probabilistic model fusion approach named Naive Adaptive Probabilistic Sensor (NAPS).
• NAPS combines machine learning paradigms built around bagging algorithms for enhanced data handling.

Main Results:

• Achieved a significant overall performance improvement with NAPS, reaching an accuracy of 95.29% in detecting human task errors.
• Demonstrated a negligible performance drop (93.93% accuracy) even with ambiguous ground truth labels.
• Outperformed other methodologies, which achieved only 64.91% accuracy.

Conclusions:

• NAPS effectively overcomes challenges posed by experimental data limitations in human performance research.
• The modular design of NAPS facilitates future sensor integration and handling of conflicting ground truth data.
• This probabilistic model fusion approach sets a foundation for human-centric modeling systems reliant on human state prediction.