A research coding method to evaluate a smoking cessation model for training residents-A preliminary report

Area of Science:

• Medical Education
• Behavioral Science
• Health Communication

Background:

• Effective resident training in smoking cessation is crucial for public health.
• Standardized methods are needed to assess the quality of resident-physician communication in cessation counseling.
• Existing coding schemes may not fully capture the nuances of patient-centered cessation interventions.

Purpose of the Study:

• To develop and validate a reliable coding method for assessing resident skills in a simulated patient smoking cessation model.
• To establish inter-coder reliability for a new coding scheme designed for motivational interviewing in smoking cessation.
• To refine a coding system for evaluating key components of resident-led smoking cessation interventions.

Main Methods:

• Two trained coders analyzed 161 videotaped resident-simulated patient interactions.
• A subset of 33 interactions (20%) was double-coded to establish reliability using Cohen's Kappa and percent agreement.
• The coding scheme focused on residents' skills in educating, informing, motivating, goal setting, negotiation, patient-centeredness, and emotional support.

Main Results:

• The coding method demonstrated high reliability, with overall Cohen's Kappa at 0.84 and percent agreement at 93%.
• Reliability for individual variables ranged from 0.73 to 0.87, and item agreement ranged from 82% to 100%.
• The initial 50 items were refined to 28 during the training and analysis process.

Conclusions:

• A highly reliable coding method for a smoking cessation training model was successfully developed.
• The method, weighted to emphasize key teaching elements, is recommended for research focusing on patient-centered communication, emotional support, and treatment planning.
• This approach offers a robust tool for evaluating and improving resident training in smoking cessation interventions.