Real time interrater reliability of a novel musculoskeletal readiness screening tool

Military service members receive regular screenings for a variety of health conditions, but a field-expedient and military-specific screening tool that identifies an individual's risk for injury has not yet been identified. The purpose of this study is to describe the conduct of a novel musculoskeletal readiness screening tool (MRST) and evaluate the real-time interrater reliability of the MRST when scored by raters with differing levels of medical experience.

This study included a convenience sample of 40 active duty military participants (30 male, 10 female, mean age 29.3±6.9 years) without any current musculoskeletal injury or pain at the time of enrollment. The MRST consisted of 5 physical performance tests and one self-report question as follows: (1) weight-bearing lunge (WBL), (2) overhead squat, (3) closed kinetic chain upper extremity stability test (CKCUEST), (4) eyes closed forward step down, (5) repeated tuck jump, and (6) individual perceived level of risk for MSK injury. Three raters (a board certified physical therapist with 15 years of experience, a physical therapy student with less than one year didactic training, and a physical therapy technician with approximately 10 years of experience) independently scored each event as 0, 1, or 2 based on the quality of the participant's performance. This scoring system allows for a cumulative score ranging from 0 to 12, with lower scores thought to indicate higher risk for future injury. Descriptive, reliability, and chance-corrected agreement statistics were calculated using IBM SPSS. This study was approved by the Brooke Army Medical Center Institutional Review Board at Fort Sam Houston, Texas.

The mean composite MRST score for all graders was 7.79±1.41. Among all 3 raters the overall reliability was moderate (ICC (2,1)=0.75 (0.62, 0.85)). Chance-corrected agreement values for the individual events ranged from slight to almost perfect as follows: WBL (κ=0.33-0.44), overhead squat (κ=0.57-0.65), CKCUEST (κ=0.89-1.0), eyes-closed forward step down (κ=0.10-0.42), repeated tuck jump (κ=0.39-0.61), individual perceived level of risk for MSK injury (κ=1.0).

The MRST showed moderate interrater reliability for the overall composite score with varied levels of agreement for individual events scores. Future research should investigate test-retest reliability and interrater reliability among medical personnel from different disciplines.