Evaluation of a Training Program to Improve Organizational Capacity for Health Systems Analytics

Area of Science:

• Biomedical and Health Informatics (BMHI)
• Data Science Education
• Professional Development

Background:

• The Leadership in Analytics and Data Science (LEADS) course was developed to address the need for advanced analytics skills among working BMHI professionals.
• The 6-month program aimed to enhance expertise in analytics, enterprise applications, and data stewardship, fostering a community of practice.

Purpose of the Study:

• To evaluate the effectiveness of the LEADS course using the Kirkpatrick Model.
• To assess improvements in participants' knowledge, analytics capabilities, practice patterns, and professional networks.

Main Methods:

• The Kirkpatrick Model was employed, assessing pre- and post-course metrics including knowledge, capabilities, goals, practice, and network size.
• Statistical analysis included Wilcoxon signed rank tests for significance and z-statistic for effect sizes.

Main Results:

• 96% of participants would recommend the LEADS course.
• Significant improvements were reported in Epic data warehousing (p=0.017), IRB policy (p=0.005), and data stewardship (p=0.007).
• Participant professional networks doubled on average, mirroring improvements in self-reported capabilities.

Conclusions:

• The LEADS course successfully met its objectives of expanding BMHI knowledge, skills, and professional networks.
• This evaluation is the first formal effectiveness assessment of such a course for BMHI professionals at a major academic medical center.
• The LEADS course serves as a valuable template for continuing education in BMHI.