Neural Signatures of Interface Errors in Remote Agent Manipulation

The manipulation of remote agents such as robotic arms in remote surgery or in BCI-wheelchair control are prone to errors. Some of these are related to user intent misclassification or other interface system errors, which lead to an incorrect movement. Here we focused on errors originating from unpredicted interface movements violating user intent and producing sensory conflicts. In addition, we examined effects of incongruent/congruent sensory stimuli induced by interface errors, focusing on haptic and visual cues in the system. The overarching goal was to identify the prototypical patterns of electroencephalogram (EEG) error signals associated with two types of interface errors rising when the visual and proprioceptive feedback are congruent or incongruent. For purposes of comparison validity, both types of errors were recorded in the same 3D virtual game environment. The comparison of congruent and incongruent interface errors revealed significant and marginally significant differences in EEG potentials with respect to profile, latencies, scalp distribution and sources. Different EEG time-frequency combinations had high power content. Incongruence between visual and proprioceptive feedback in interface errors not only elicited distinct EEG signal characteristics, but also produced a marginally significant Stroop effect. Incongruency in visuo-haptic feedback modalities cause a delayed user response. This effect is of major importance for the design of controlling interfaces and can provide designers with crucial information when aiming to control human response time.