Verified Research Productivity Among Matched Orthopaedic Surgery Residency Applicants: Establishing a National...

The shift of the United States Medical Licensing Examination (USMLE) Step 1 to pass/fail scoring in 2022 elevated research productivity's role in orthopaedic surgery residency applications. With the 2027 Electronic Residency Application Service (ERAS) cycle reforms limiting research entries to peer-reviewed publications and consolidating repeated presentations, establishing verifiable research productivity baselines is critical. This study provides the earliest comprehensive verification of PubMed-indexed publications across an entire national orthopaedic match cohort, quantifying disparities between self-reported and verified activities and establishing benchmarks to assess future reform impacts.

Cross-sectional bibliometric analysis examined first-year orthopaedic surgery residents who matched in the 2023-2024 National Resident Matching Program (NRMP) or military cycle (N=929, 97% of matched positions). A novel automated disambiguation system integrated institutional affiliation matching, name uniqueness, and temporal filtering to verify publications prior to September 24, 2023, yielding 4,774 verified publications from 46,897 extracted. Programs were assigned to six tiers based on composite scores integrating established program rankings with contemporary research productivity. Tier 1 (10 programs, score ≥ 50) and Tier 6 (76 programs, score < 2) represented the highest and lowest tiers. Statistical analyses employed Kruskal-Wallis and Mann-Whitney U tests for group comparisons, receiver operating characteristic (ROC) analysis for threshold determination, and incremental benefit analysis to assess diminishing returns.

Of 958 identified positions, 929 residents were collected (773 with complete data). The mean verified publication count was 5.5 ± 9.0 (median 3.0; range 1-139), representing 23% of mean self-reported outputs (23.8) in the 2024 NRMP Charting Outcomes report; 77% comprised non-peer-reviewed activities. Clinical research constituted 46% (n=2187) of publications, with original research accounting for 75% (n=3566). Residents were first, second, or third author on 68.2% (n=3254) of publications. Higher-tier programs had significantly greater publication counts (Tier 1: 9.7 ± 8.8 vs. Tier 6: 3.7 ± 6.7; H=100.54, p < 0.0001). ROC analysis identified an optimal threshold of two to four publications (AUC = 0.66) for predicting high-tier matches. Incremental benefit analysis revealed the largest marginal gain between one and two publications (mean tier improvement=0.55; p < 0.001), with diminishing returns thereafter, and a secondary benefit peak at seven publications (0.80; p < 0.05).

This study establishes the earliest verified baseline for research productivity in an orthopaedic surgery match cohort, documenting substantial inflation between self-reported activities and peer-reviewed publications. Positioned between the 2022 Step 1 pass/fail transition and the 2027 ERAS application reforms, these data provide benchmark metrics for evaluating the impact of future verification systems and application changes. Identification of optimal thresholds (two to four papers) and quantification of incremental benefits offer evidence-based guidance for students, advisors, and program directors. Our novel verification methodology, achieving 83.2% (n=773 residents) cohort coverage, establishes a reproducible framework applicable across specialties for transparent scholarly productivity assessment.