Exploring the knowledge demands of a pharmacology assessment using Legitimation Code Theory

Designing appropriate assessments is essential in driving student learning. As pharmacology lecturers we aspire to create teaching and learning activities which allow students to gain both depth of understanding and ability to transfer their knowledge. Thus, carefully designed assessments are a key requirement to align with educator's intentions and students' needs.

To investigate whether Legitimation Code Theory and one of its three dimensions, Semantics, is an effective tool to reveal the types of knowledge demands embedded in a dental pharmacology assessment.

Three independent researchers developed a 'translation device' to code assessment demands in terms of semantic gravity (the context-dependence of meaning) and semantic density (the degree of complexity of meaning). Coding was plotted on the Cartesian plane, the four quadrants representing how semantic gravity and semantic density fluctuated throughout individual questions.

Using our translation device the semantic gravity and semantic density coding analysis identified a range of both abstraction and complexity embedded in dental pharmacology summative questions. Coding resulted in Kappa interrater values of 1.0 for semantic gravity and ranging from 0.85 to 1.0 for semantic density. Compulsory clinical case studies had knowledge demands mainly embedded in strong semantic gravity (SG+) and weak semantic density (SD-) while the optional questions had knowledge demands with a wider semantic range and were distributed throughout three quadrants of the semantic plane.

This study introduces a semi-objective approach to analyse and uncover the knowledge demands embedded in pharmacology assessments, ensuring alignment with students' learning outcomes and fostering awareness of the knowledge required for exam success.