A systematic review of major evaluation metrics for simulator-based automatic assessment of driving after stroke

Background: Simulator-based driving assessments (SA) have recently been used and studied for various purposes, particularly for post-stroke patients. Automating such assessment has potential benefits especially on reducing financial cost and time. Nevertheless, there currently exists no clear guideline on assessment techniques and metrics available for SA for post-stroke patients. Therefore, this systematic review is conducted to explore such techniques and establish guidelines for evaluation metrics. Objective: This review aims to find: (a) major evaluation metrics for automatic SA in post-stroke patients and (b) assessment inputs and techniques for such evaluation metrics. Methods: The study follows the PRISMA guideline. Systematic searches were performed on PubMed, Web of Science, ScienceDirect, ACM Digital Library, and IEEE Xplore Digital Library for articles published from January 1, 2010, to December 31, 2023. This review targeted journal articles written in English about automatic performance assessment of simulator-based driving by post-stroke patients. A narrative synthesis was provided for the included studies. Results: The review included six articles with a total of 239 participants. Across all of the included studies, we discovered 49 distinct assessment inputs. Threshold-based, machine-learning-based, and driving simulator calculation approaches are three primary types of assessment techniques and evaluation metrics identified in the review. Discussion: Most studies incorporated more than one type of input, indicating the importance of a comprehensive evaluation of driving abilities. Threshold-based techniques and metrics were the most commonly used in all studies, likely due to their simplicity. An existing relevant review also highlighted the limited number of studies in this area, underscoring the need for further research to establish the validity and effectiveness of simulator-based automatic assessment of driving (SAAD). Conclusions: More studies should be conducted on various aspects of SAAD to explore and validate this type of assessment.