A validation study on the use of intra-operative video recording as an objective assessment tool for core ENT surgery

To investigate the reliability and validity of remote scoring a video assessment of a core ENT surgical procedure (myringotomy and grommet insertion) and its suitability as an objective tool for assessing the technical skills of ENT surgeons.

Single-blinded (raters) video assessment.

Tertiary Care University Hospital.

Consultant and trainee [Specialty Registrar (StR) and Core Trainee (CT)] ENT surgeons performing a total of 30 consecutive index procedures.

To determine the construct validity and the reliability of video assessment scoring of myringotomy and grommet insertion by two raters at ENT Consultant level with a subspecialty interest in Otology. To measure the performance (by rating) of participants compared to operative time.

A strong correlation between scores by the two blinded raters was demonstrated (ρ = 0.748; P < 0.001). Median scores (/45) for each group were as follows: CT 25.5 (IQR 21.13-31.25), StR 33 (IQR 24.88-35) and Consultant 40 (IQR 35.38-42.63). Kruskal-Wallis test analysis showed statistically significant mean rank scores between the three different levels of experience (H = 12.77, P = 0.002). Multiple group comparisons indicated a significant difference between CT and Consultant groups (P < 0.001) and StR and Consultant groups (P = 0.007). Analysis of the time taken between the experience groups demonstrated a difference (H = 8.689, P = 0.013) although individual intergroup comparisons indicated this was only significant between CT and Consultant groups (P = 0.004). There was a significant negative correlation (ρ = -0.842; P < 0.001) between time taken for procedure and score achieved.

Video assessment of myringotomy and ventilation tube insertion may represent a valid, feasible tool for use in summative and formative assessments of trainee ENT surgeons. Remote scoring of assessment procedures minimises bias and enables blinding of raters. ENT is well positioned to benefit from video assessment due to the high number of surgical procedures within the specialty that are performed utilising digital technology.