A method for assessing fidelity of delivery of telephone behavioral support for smoking cessation

Behavioral support for smoking cessation is delivered through different modalities, often guided by treatment manuals. Recently developed methods for assessing fidelity of delivery have shown that face-to-face behavioral support is often not delivered as specified in the service treatment manual. This study aimed to extend this method to evaluate fidelity of telephone-delivered behavioral support.

A treatment manual and transcripts of 75 audio-recorded behavioral support sessions were obtained from the United Kingdom's national Quitline service and coded into component behavior change techniques (BCTs) using a taxonomy of 45 smoking cessation BCTs. Interrater reliability was assessed using percentage agreement. Fidelity was assessed by comparing the number of BCTs identified in the manual with those delivered in telephone sessions by 4 counselors. Fidelity was assessed according to session type, duration, counselor, and BCT. Differences between self-reported and actual BCT use were examined.

Average coding reliability was high (81%). On average, 41.8% of manual-specified BCTs were delivered per session (SD = 16.2), with fidelity varying by counselor from 32% to 49%. Fidelity was highest in pre-quit sessions (46%) and for BCT "give options for additional support" (95%). Fidelity was lowest for quit-day sessions (35%) and BCT "set graded tasks" (0%). Session duration was positively correlated with fidelity (r = .585; p < .01). Significantly fewer BCTs were used than were reported as being used, t(15) = -5.52, p < .001.

The content of telephone-delivered behavioral support can be reliably coded in terms of BCTs. This can be used to assess fidelity to treatment manuals and to in turn identify training needs. The observed low fidelity underlines the need to establish routine procedures for monitoring delivery of behavioral support.