The validation of three human reliability quantification techniques--THERP, HEART and JHEDI: Part III--Practical...

This is the third paper in a series of three dealing with the detailed investigation of the empirical validity of three human reliability assessment (HRA) techniques. The first paper introduced the need for validation and specified the three techniques most requiring validation. The second paper detailed the results of an extensive independent validation experiment. This experimental validation involved 30 UK assessors using the techniques THERP, HEART and JHEDI (10 assessors per technique) to estimate the human error probabilities (HEPs) for 30 nuclear power and reprocessing (NP&R) tasks. The results for all three techniques were positive in terms of significant correlations, and general precision levels of 72% of all HEP estimates within a factor of 10 of the true value (unknown to the assessors). These results lend support to the empirical validity of these techniques in particular, and to HRA in general. However, the results were not all positive. In particular the consistency of usage of the techniques was variable. Additionally, subjects were generally not good at knowing their own uncertainty, i.e. they were not able to accurately predict when they were accurate nor when they were inaccurate. This desirable parameter is known as calibration, and the results from the validation suggested that subjects were not well-calibrated. This paper aims to determine how consistency of usage can be improved and to discern whether certain task types are, in practice, not well-assessed by the techniques, and hence are effectively currently beyond these techniques' abilities. Such information is aimed at aiding the HRA practitioner, or the ergonomist, interested in using these techniques. Recommendations for improving calibration are also discussed in this paper. A subsidiary but important focus of this paper is of a more fundamental nature, and of more general interest to the ergonomist. It concerns the validity of the techniques from an error reduction perspective. Currently these techniques may be used to identify how to reduce error probability, which is generally (in the qualitative sense) within the domain of ergonomics. One major mechanism for HRA-based error reduction is the utilisation of Performance Shaping Factor (PSF) information. This paper considers the validity of these PSF as ergonomics constructs. Drawing results from the validation exercise, it is seen how different PSF can be applied to the same scenario and can result in the same error probability, but will result in different error reduction guidance. It is therefore recommended that error reduction guidance must be based on a composite analysis of the results of the task, error identification and quantification analyses, with most weighting given to the qualitative analyses.