A Skills Assessment Pathways-Based Program Assessment Approach in Multidisciplinary Graduate Health Informatics

Multidisciplinary graduate education programs are hard to assess because of interdependent competencies. Students in these programs come with diverse disciplinary undergraduate degrees, and it is critical to identify knowledge gaps among these diverse learner groups to provide support to fill these gaps. Health Informatics (HI) is a multidisciplinary field in which health, technology, and social science knowledge are foundational to building HI competencies. In 2017, the American Medical Informatics Association identified ten functional domains in which HI competencies are divided. Using pre/post-semester knowledge assessment surveys of graduate students (n=60) between August 2021 to May 2022 in one of the largest graduate HI programs in the United States, we identified courses (n=9) across the curriculum that help build HI-specific competencies. Using statistical analysis, we identified three skills pathways by correlating knowledge gained with course learning objectives and used this to modify the curriculum over four semesters. These skills pathways are connected through one or two courses, where students can choose electives or, in some instances, course modules or assignments that link the skills pathways. Moreover, there is a statistically significant difference in how students gain these skills depending on their prior training, even though they take the same set of courses. Gender and other demographics did not show statistical differences in skills gained. Additionally, we found that research assistantships and internships/practicums provide additional skills not covered in our HI curriculum. Our program assessment methodology and resulting curricular changes might be relevant to HI and other multidisciplinary graduate training programs.