Prepublication Podcast Peer Review: Summary of an Innovation

Podcasts are a popular educational tool for health professions, known for their availability and flexibility. Skeptics criticize podcasts for often absent editorial oversight, which can threaten quality. The purpose of this pilot project was to introduce and standardize prepublication peer review for a pediatric podcast.

The Peer Review Tool was adapted to incorporate reviewer clinical expertise and subjective feedback in a prepublication format. A standardized prepublication peer review process mirrored refereed journals. Volunteer nurse practitioner faculty wrote and reviewed a pediatric podcast series. The article authors and an external reviewer also independently scored episodes. Interrater reliability was evaluated using percent agreement and intraclass correlation coefficient (3,4). Numeric review scores were displayed publicly online in podcast show notes.

An 11-episode podcast series was written and reviewed by 23 nurse practitioners. Each reviewer was matched with an episode based on their identified clinical expertise. Most episodes (n = 10; 90.9%) were published with the highest possible score. Percent agreement was very good between two peer reviewers and article authors (90.9%) when compared with the external reviewer (72.7% and 81.8%, respectively). Intraclass correlation coefficient (3,4) was 0.45, reflecting moderate reliability because of limited variability of scores and ceiling effect.

A standardized approach to prepublication peer review consistently produced podcasts that met established high-quality standards. Any podcast producer can replicate this process to achieve guideline calls to meet quality standards. More research is needed to adapt prepublication tools to increase rigor and content integrity, train users on their use, and improve transparency for listeners.