Related Experiment Video
Updated: Jun 23, 2025
Objectively Assessing Sports Concussion Utilizing Visual Evoked Potentials
Published on: April 27, 2021
Testing the Validity and Reliability of a Standardized Virtual Examination for Concussion
Alani I Jack1, Helena T Digney1, Carter A Bell1
1Department of Neurology (AIJ, HTD, CAB, SNG, LJB, SLG, NAB), New York University Grossman School of Medicine, NY; Department of Rehabilitation Sciences (JIM, GTS), School of Public Health and Health Professions, State University of New York at Buffalo; UBMD Department of Orthopaedics and Sports Medicine (MNH, JJL) and Department of Psychiatry (BSW), Jacobs School of Medicine and Biomedical Sciences, State University of New York at Buffalo; Departments of Population Health (LJB) and Ophthalmology (LJB, SLG), New York University Grossman School of Medicine, NY; and Department of Neurology (DMT), Lenox Hill Hospital at Northwell Health, New York, NY.
Background And Objectives:
We determined inter-modality (in-person vs telemedicine examination) and inter-rater agreement for telemedicine assessments (2 different examiners) using the Telemedicine Buffalo Concussion Physical Examination (Tele-BCPE), a standardized concussion examination designed for remote use.
Methods:
Patients referred for an initial evaluation for concussion were invited to participate. Participants had a brief initial assessment by the treating neurologist. After a patient granted informed consent to participate in the study, the treating neurologist obtained a concussion-related history before leaving the examination room. Using the Tele-BCPE, 2 virtual examinations in no specific sequence were then performed from nearby rooms by the treating neurologist and another neurologist. After the 2 telemedicine examinations, the treating physician returned to the examination room to perform the in-person examination. Intraclass correlation coefficients (ICC) determined inter-modality validity (in-person vs remote examination by the same examiner) and inter-rater reliability (between remote examinations done by 2 examiners) of overall scores of the Tele-BCPE within the comparison datasets. Cohen's kappa, κ, measured levels of agreement of dichotomous ratings (abnormality present vs absent) on individual components of the Tele-BCPE to determine inter-modality and inter-rater agreement.
Results:
For total scores of the Tele-BCPE, both inter-modality agreement (ICC = 0.95 [95% CI 0.86-0.98, p < 0.001]) and inter-rater agreement (ICC = 0.88 [95% CI 0.71-0.95, p < 0.001]) were reliable (ICC >0.70). There was at least substantial inter-modality agreement (κ ≥ 0.61) for 25 of 29 examination elements. For inter-rater agreement (2 telemedicine examinations), there was at least substantial agreement for 8 of 29 examination elements.
Discussion:
Our study demonstrates that the Tele-BCPE yielded consistent clinical results, whether conducted in-person or virtually by the same examiner, or when performed virtually by 2 different examiners. The Tele-BCPE is a valid indicator of neurologic examination findings as determined by an in-person concussion assessment. The Tele-BCPE may also be performed with excellent levels of reliability by neurologists with different training and backgrounds in the virtual setting. These findings suggest that a combination of in-person and telemedicine modalities, or involvement of 2 telemedicine examiners for the same patient, can provide consistent concussion assessments across the continuum of care.
