Procedural trial generation for cognitive assessment: An executive function planning task

We introduce an approach for the computational generation of new trials to support the design of cognitive assessments and demonstrate the methodology through a novel executive functioning planning task. Drawing inspiration from procedural content generation (PCG; i.e., computationally creating new content based on a defined set of rules) in gaming, we created a scalable and adaptable method for generating a variety of experimental trials. Our exemplary task, influenced by traditional cognitive measures and digital puzzle games, involves moving colored blocks to form matches and clear the screen, assessing planning. The task development process is detailed from designing mockups through to coding, making the task accessible online, and conducting usability and functional testing. The procedural generation technique, used to automatically generate trials for the task, followed a generate-and-test methodology. The process began with the pseudo-random creation of puzzles (i.e., generate), then a solving algorithm explored the generated configurations and retained only the solvable puzzles (i.e., test). This approach automatically produced many diverse trials for our task. We discuss future directions for PCG in cognitive assessment, including the potential exploration of advanced algorithms to further optimize trial generation. We also provide guidelines for digital cognitive task development and procedural trial generation in cognitive assessment, including incorporating psychometric validation and adaptive difficulty, to facilitate the generation of many high-quality trials for a broad range of applications, including cognitive intervention studies and cognitive ecological momentary assessment.