Gaze-based detection of mind wandering during audio-guided panorama viewing

Unlike classic audio guides, intelligent audio guides can detect users' level of attention and help them regain focus. In this paper, we investigate the detection of mind wandering (MW) from eye movements in a use case with a long focus distance. We present a novel MW annotation method for combined audio-visual stimuli and collect annotated MW data for the use case of audio-guided city panorama viewing. In two studies, MW classifiers are trained and validated, which are able to successfully detect MW in a 1-s time window. In study 1 (n = 27), MW classifiers from gaze features with and without eye vergence are trained (area under the curve of at least 0.80). We then re-validate the classifier with unseen data (study 2, n = 31) that are annotated using a memory task and find a positive correlation (repeated measure correlation = 0.49, p < 0.001) between incorrect quiz answering and the percentage of time users spent mind wandering. Overall, this paper contributes significant new knowledge on the detection of MW from gaze for use cases with audio-visual stimuli.