Issues in developing valid assessments of speech pathology students' performance in the workplace

Area of Science:

• Health Professions Education
• Speech Pathology Assessment
• Workplace Learning Evaluation

Background:

• Workplace learning is crucial for speech pathology training, but validated assessments are difficult to develop.
• Key challenges include defining judgment in assessment, measuring quality, and linking assessment to learning.
• Effective assessment must track competence development and include both specific and general skills.

Purpose of the Study:

• To review conceptual issues in designing valid and reliable workplace-based assessments.
• To share learnings on quality assessment and validation of competency-based performance assessments.
• To describe the development and validation of a national assessment tool.

Main Methods:

• Conducted a literature review on conceptualizing, designing, and validating workplace-based assessments.
• Identified key design factors and integrated them into a tool development and validation cycle.
• Utilized Rasch analysis for statistical validation of the assessment tool.

Main Results:

• Presents an overview of factors for designing and validating workplace assessment tools.
• Details the development and national validation of the COMPASS tool for speech pathology programs.
• Demonstrates the utility of Rasch analysis for establishing assessment tool validity and reliability.

Conclusions:

• Successfully developed the world's first valid and reliable national assessment tool for speech pathology performance.
• Highlights the importance of addressing conceptual and design issues in assessment development.
• The COMPASS tool provides a standardized method for evaluating speech pathology competence.