Inter-rater reliability of the Hayes Ability Screening Index in a sample of Australian prisoners

Reliable ascertainment of intellectual disability (ID) is important to identify those with special needs, in order for those needs to be met in the criminal justice system. Although the Hayes Ability Screening Index (HASI) is valid and widely used for the identification of possible ID, the risk of inter-rater bias between researchers when scoring the HASI has not yet been established. The current paper estimates the inter-rater reliability of the HASI in a sample of Indigenous and non-Indigenous prisoners in Western Australia.

We estimated intra-class correlation coefficients (ICC) for the consistency of agreement among three blinded raters using a two-way random-effects model assessing the inter-rater agreement of the HASI. Kappa was also estimated for the dichotomous HASI screening threshold outcome between the raters.

The HASI exhibited very good within-subject consistency of agreement for Section B (ICC = 0.95; 95%CI:0.94-0.96), Section C (ICC = 0.97; 95%CI: 0.96-0.98) and Section D (ICC = 0.90; 95%CI: 0.87-0.92) subscales and for the total scaled score (ICC = 0.97; 95%CI: 0.96-0.98). The inter-rater reliability of the dichotomous adult ID screening threshold (<85) was also very good (Kappa = 0.95).

The current study provides new evidence that the HASI has a low risk of bias from between-rater scoring and can be reliably scored by both non-clinicians and clinicians with little training, when administered in prison settings. Pre-scoring training should focus on the more subjective 'clock-drawing' section, in order to maximise inter-rater reliability.