Related Experiment Video
Updated: Feb 22, 2026
Three-Dimensional Finger Motion Tracking during Needling: A Solution for the Kinematic Analysis of Acupuncture Manipulation
Published on: October 28, 2021
Surgical Skills Feedback and myTIPreport: Is There Construct Validity?
AnnaMarie Connolly1, Anita Blanchard, Alice Goepfert
1Departments of Obstetrics and Gynecology, University of North Carolina, Chapel Hill, North Carolina, University of Chicago, Chicago, Illinois, University of Alabama, Birmingham, Birmingham, Alabama, University of Pittsburgh, Pittsburgh, Pennsylvania, Mountain Area Health Education Center, Asheville, North Carolina, and Northwestern University, Chicago, Illinois.
Objective:
To initiate construct validity testing of myTIPreport for procedural skill assessment in a prospective multicenter evaluation study.
Methods:
Teachers and learners from a convenience-based site selection of obstetrics and gynecology (OBGYN) and female pelvic medicine and reconstructive surgery (FPMRS) training programs performed procedural assessments in myTIPreport. The specifically defined 5-point Dreyfus rating scale describing ability levels from novice to expert was used. Defined as the degree to which a test or measure assesses what it was designed to measure, construct validity of myTIPreport was tested by comparing the medians of procedure-specific overall assessments, by both teachers and learners themselves, of senior learners with junior learners. To minimize type I error, comparisons were performed only when a threshold of 10 or greater feedback encounters per learner group was met. Correlation of teacher assessments and learner self-assessments was examined for myTIPreport.
Results:
From November 2014 to May 2016, 12 OBGYN and 7 FPMRS training programs participated. There were 440 learners and 443 teachers. Feedback was recorded on 5,093 surgical procedures; 4,567 for OBGYN residents and 526 for FPMRS fellows. Each OBGYN procedure had two categories of teacher and learner assessments comparing postgraduate year (PGY)-4 with PGY-1 learner performance. This yielded 48 possible assessment comparisons for the included 24 OBGYN procedures. In all, 28 of these 48 (58%) met the threshold number of observations per learner group. In 28 of these 28 (100%) comparison categories, PGY-4s rated significantly higher than PGY-1s. Similarly, in 16 of 18 (89%) comparison categories meeting inclusion criteria, FPMRS PGY-7s rated significantly higher than FPMRS PGY-5s. Strong correlation was noted of teacher assessments and learner self-assessments in myTIPreport with a Spearman correlation coefficient of 0.89 (P<.001).
Conclusion:
As noted for the majority of compared teacher assessments and learner self-assessments, myTIPreport appeared to detect differences between senior and junior learners. These data support the emerging construct validity of myTIPreport for procedural skills assessment.
