Development of fidelity of delivery and enactment measures for interventions in communication disorders

This study was part of a process evaluation for a single-blind, randomized controlled pilot study comparing Better Conversations with Primary Progressive Aphasia (BCPPA), an approach to communication partner training, with no speech and language therapy treatment. It was necessary to explore fidelity of delivery (delivery of intervention components) and intervention enactment (participants' use of intervention skills in the form of conversation behaviours comprising facilitators, that enhance the conversational flow, and barriers, that impeded the flow of conversation). This study aimed to: (1) Outline an adapted methodological process that uses video observation, to measure both fidelity of delivery and enactment. (2) Measure the extent to which the BCPPA pilot study was delivered as planned, and enacted.

Observational methods were used alongside statistical analysis to explore the fidelity of intervention and enactment using video recordings obtained from the BCPPA pilot study.

A 5-step methodology, was developed to measure fidelity of delivery and enactment for the BCPPA study using video-recorded data. To identify delivery of intervention components, a random sample of eight video recorded and transcribed BCPPA intervention sessions was coded. To examine the enactment of conversation behaviours, 108 transcribed 10 -min-video recorded conversations were coded from 18 participants across the control and intervention group.

Checklists and guidelines for measurement of fidelity of treatment delivery and coding spreadsheets and guidelines for measurement of enactment are presented. Local collaborators demonstrated 87.2% fidelity to the BCPPA protocol. Participants in the BCPPA treatment group increased their use of facilitator behaviours enacted in conversation from a mean of 13.5 pre-intervention to 14.2 post-intervention, whilst control group facilitators decreased from a mean of 15.5 to 14.4, over the same timescale.

This study proposes a novel and robust methods, using video recorded intervention sessions and conversation samples, to measure both fidelity of intervention delivery and enactment. The learnings from this intervention are transferable to other communication interventions.