Beyond Cronbach's Alpha in Dental Education Assessments: Introducing Composite Indices for Enhanced Reliability and...

Dental educators design assessments to evaluate multiple competencies across diverse topics using items of varied difficulty levels, topical coverage, and mixed scoring weights, making them inherently multidimensional and criterion-referenced. In practice, however the reliability of such assessments is often analysed using Cronbach alpha, one of the most widely used indices of internal consistency, despite its underlying assumptions of unidimensionality and equal item contribution.

Using a dataset of more than 200 000 item scores, we computed Cronbach's alpha at various grouping levels and analysed it alongside to other psychometric indices.

(1) Dependency of Cronbach's alpha on number of items, regardless of their underlying construct; (2) At the exam level, where item diversity is intentional, this practice can penalize heterogeneity by obscuring item-level strengths; (3) Even at finer levels, such as within a topic, the assumptions begin to break down; and (4) In contrast, the proposed composite indices provide a more consistent and diagnostically informative evaluation of assessment quality.

Our proposed framework: one at the item level, the Composite Item Index (CII) and another at the exam level, the Adjusted Composite Exam Index (ACEI) captures item difficulty variation along with topic coverage. Interpretation of 122 assessments using the proposed framework demonstrates greater stability, diagnostic value, and educational relevance compared to Cronbach's alpha. Monitoring CII and ACEI throughout the assessment workflow provides diagnostic insight beyond traditional reliability indices and supports psychometrically sound and balanced assessments essential for competency-based education. Together, they offer a construct-aware, multi-metric framework for quality assurance.