Related Experiment Video
Updated: Jan 15, 2026
Objectively Assessing Sports Concussion Utilizing Visual Evoked Potentials
Published on: April 27, 2021
Visible signs of possible concussion at the FIFA World Cup Qatar 2022™: Independent review and inter-rater
Andreas Serner1, Margot Putukian2, Ruben Echemendia3
1FIFA Medical, Fédération Internationale de Football Association (FIFA), Switzerland.
Objectives:
To report the frequency of head impacts and associated visible signs of possible concussion, and their inter-rater reliability at the FIFA World Cup Qatar 2022™.
Design:
Observational and inter-rater reliability study.
Methods:
Fifteen visible signs of potential sport-related concussion, grouped into "red flags", "yellow flags", and "other signs", were recorded live by FIFA injury spotters during the tournament. Post-tournament all matches were reviewed for visible signs by independent injury spotters. Agreement was assessed using Fleiss kappa (κ) and Prevalence And Bias Adjusted Kappa.
Results:
FIFA spotters registered 172 head impacts live, while the post-tournament independent spotters registered 206 (67 % positive agreement). Both groups observed five incidents with at least one "red flag", though these were not identical, resulting in moderate agreement (Kappa (K) = 0.58, 0.41 to 0.76). Agreement for each "red flag" varied from poor to almost perfect (K = -0.01 to 1.00). Incidents with "yellow flags" showed poor agreement (K = -0.46, -0.64 to -0.29), primarily due to differences in scoring "slow to get up", while signs "face or scalp injury" and "lying motionless" had moderate to substantial agreement (K = 0.58 to 0.66). Inter-rater reliability of each visible sign varied from poor to substantial (K = -0.01 to 0.75), with low prevalence of some signs directly influencing kappa values. For the majority of visible signs, results of the prevalence-adjusted-and-bias-adjusted kappa showed almost perfect (K = 0.81 to 1.00) agreement and inter-rater reliability.
Conclusions:
Visible signs of concussion categorized as "red flags" were infrequent, resulting in uncertain reliability. The frequency and reliability of visible signs categorized as "yellow flags" and "other signs" varied, with the visible signs "clutching of the head" and "slow to get up" occurring so frequently that their utility for identifying possible concussion in football is questionable. Further research should focus on investigating specificity of the visible signs in larger studies with surveillance and clinical follow-up from multiple competitions (seasons).
