A consensus-based gold standard for the evaluation of mass casualty triage systems

Accuracy and effectiveness analyses of mass casualty triage systems are limited because there are no gold standard definitions for each of the triage categories. Until there is agreement on which patients should be identified by each triage category, it will be impossible to calculate sensitivity and specificity or to compare accuracy between triage systems.

To develop a consensus-based, functional gold standard definition for each mass casualty triage category.

National experts were recruited through the lead investigators' contacts and their suggested contacts. Key informant interviews were conducted to develop a list of potential criteria for defining each triage category. Panelists were interviewed in order of their availability until redundancy of themes was achieved. Panelists were blinded to each other's responses during the interviews. A modified Delphi survey was developed with the potential criteria identified during the interview and delivered to all recruited experts. In the early rounds, panelists could add, remove, or modify criteria. In the final rounds edits were made to the criteria until at least 80% agreement was achieved.

Thirteen national and local experts were recruited to participate in the project. Six interviews were conducted. Three rounds of voting were performed, with 12 panelists participating in the first round, 12 in the second round, and 13 in the third round. After the first two rounds, the criteria were modified according to respondent suggestions. In the final round, over 90% agreement was achieved for all but one criterion. A single e-mail vote was conducted on edits to the final criterion and consensus was achieved.

A consensus-based, functional gold standard definition for each mass casualty triage category was developed. These gold standard definitions can be used to evaluate the accuracy of mass casualty triage systems after an actual incident, during training, or for research.