Development of the trainer skill evaluation scale for metacognitive training.

Area of Science:

• Psychology
• Educational Psychology
• Skill Assessment

Background:

• Metacognitive Training (MCT) is crucial for cognitive skill development.
• Assessing trainer effectiveness in MCT is essential for program quality.
• Existing tools for evaluating MCT trainer skills require further development and validation.

Purpose of the Study:

• To develop a reliable and valid rating scale for assessing metacognitive training (MCT) trainer skills.
• To determine the difficulty level of a behavioral checklist for MCT trainers.
• To examine the psychometric properties of the newly developed MCT Trainer Skills Rating Scale.

Main Methods:

• Study 1: Developed an MCT trainer skill behavior checklist and administered a questionnaire to 49 MCT trainers.
• Study 2: Utilized video-recorded training sessions for beginner and advanced trainers to assess the scale's reliability and validity.
• Calculated weighted kappa coefficients to determine inter-rater reliability.

Main Results:

• The MCT Trainer Skills Rating Scale consists of eight items with high reliability (Kappa coefficients of 0.71 and 0.73).
• Discriminant validity was confirmed, as advanced trainer scores were significantly higher than beginner trainer scores.
• The scale effectively differentiates between varying levels of trainer expertise.

Conclusions:

• The developed MCT Trainer Skills Rating Scale is a reliable and valid instrument for evaluating trainer performance.
• The scale can be used to assess trainer competence in delivering Metacognitive Training.
• Findings support the use of the scale for quality assurance and training improvement in MCT programs.