Issues in the evaluation of picture archiving and communication systems

Picture archiving and communication systems (PACS) are an example of the application of computer technology in the medical field. PACS automates image handling in a hospital and has the potential to transform the way radiology is currently performed. This paper focuses on the evaluation of the PACS technology, and considers the claims that have been made for PACS, how these claims might be turned into questions to be addressed by evaluation and the appropriate methods for the evaluation of PACS. A distinction is drawn between evaluation questions for which the hospital is the appropriate focus and those for which the patient is the appropriate focus. The preferred research design is different for hospital focused PACS evaluation and patient-focused evaluation of small scale PACS systems. A contemporaneous experimental comparison within hospitals is the preferred design for the patient-focused evaluation of small scale PACS systems. The patient-focused evaluation of large scale systems and the hospital-focused evaluation of all PACS systems could feasibly be conducted as contemporaneous experimental comparisons between hospitals but the large research costs implied by such a design almost certainly mean that non-contemporaneous, non-experimental comparisons within hospitals are more realistic. The current situation for the PACS technology is that it has potential, but as yet unproven, benefits and a large capital cost. Thus, the primary purpose of funding additional PACS implementations must be to add to the currently small body of evaluation evidence.