Overcoming barriers to evaluation of terminological systems

Evaluation of terminological systems has been demonstrated to be a complicated task. This is due to the broad range of terminological systems, their application, and the clinical contexts in which they can be applied. We propose an evaluation framework that explicitly distinguishes an application-independent description of terminological systems, methods to determine application-dependent requirements, and methods to assess the applicability. In order to support a systematic application-independent description of terminological systems, we present a categorization of characteristics, including explicit questions. The answers to these questions can be mapped to the requirements for a certain application of a terminological system. This framework aims at reducing the efforts for determining which terminological system is applicable for a certain clinical setting.