Creating synthetic patient data to support the design and evaluation of novel health information technology

To ensure that new health information technology supports its intended users, researchers and developers need to follow human-centered methods during all stages of the software development lifecycle, including early stage evaluations. These evaluations need to include realistic testing scenarios to ensure that they provide valuable and accurate feedback to system developers. However, obtaining realistic patient data to support these evaluations has many challenges, including the risk of re-identifying anonymized patients as well as the costs associated with connecting test systems with production ready clinical databases. Here we present a novel five-step process to create highly structured and realistic synthetic patient data to support the evaluation and comparison of early to middle stage health information technology prototypes. We applied this method to evaluate and compare three novel health information technology prototypes designed to support clinicians during the identification of high-priority patients when answering the question: "What patient should I see first?" Our novel approach fills an important gap in the evaluation of health information technology and assists designers in creating high-quality software that best supports its end users.