Enhancing prompt perception in dementia: a comparative study of mixed reality cue modalities

Dementia impacts millions worldwide and is challenging individuals' ability to engage in daily activities. Active living is crucial in mitigating dementia's neurodegenerative effects, yet people with dementia often struggle to initiate and complete tasks independently. Technologies offer promising solutions to engage people with dementia in activities of active living and improving their quality of life through prompting and cueing. It is anticipated that developments in sensor and wearable technologies will result in mixed reality technology becoming more accessible in everyday homes, making them more deployable. The possibility of mixed reality technologies to be programmed for different applications, and to adapt them to different levels of impairments, behaviours and contexts, will make them more scalable.

The study aimed to develop a better understanding of modalities of prompts that people with dementia perceive successfully and correctly in mixed reality environments. It investigated interactions of people with dementia with different types of visual (graphics, animation, etc.) and sound (human voice, tones, etc.) prompts in mixed reality technologies.

We used the Research through Design (RtD) method in this study. This paper describes the findings from the user research carried out in the study. We conducted observation studies with twenty-two people with dementia playing games on off-the-shelf mixed reality technologies, including both Augmented Reality (HoloLens, ArKit on iPhone) and Augmented Virtuality (Xbox Kinect and Osmo) technologies. The interactions with the technologies during the gameplay were video recorded for thematic analysis in Noldus Observer XT (version 16.0) for successful and correct perception of prompts.

A comparison of the probability estimates of correct perception of the prompts by people with dementia suggests that human voice, graphic symbols and text are the most prominently perceived modalities of prompts. Feedback prompts for every action performed by people with dementia on the technology are critical for successful perception and should always be provided in the design.

The study has resulted in recommendations and guidelines for designers to design prompts for people with dementia in mixed-reality environments. The work lays the foundation for considering mixed reality technologies as assistive tools for people with dementia, fostering discussions on their accessibility and inclusive design in technology development.