Digital Evidence: Revisiting Assumptions at the Intersection of Technology and Assessment

Area of Science:

• Health Professions Education
• Medical Informatics
• Educational Technology

Background:

• Technology integration in healthcare and health professions education is rapidly increasing.
• The use of digital evidence for assessment claims requires careful examination.
• Understanding how data is utilized is crucial for effective evaluation of learner competency.

Purpose of the Study:

• To explore the utilization of digital evidence in health professions education assessments.
• To introduce a framework of four term sets for analyzing digital data in assessments.
• To assist developers and users in understanding technology-based assessment systems.

Main Methods:

• Describing four sets of terms: primary/secondary data, structured/unstructured data, development/use, and deterministic/generative.
• Utilizing multiple examples to illustrate the application of these terms.
• Analyzing how these terms aid in evaluating learner knowledge and skills.

Main Results:

• The four term sets provide a structured approach to examining digital data in assessments.
• These terms help differentiate types of data and their application in assessment development and use.
• Examples demonstrate the practical utility of the framework for various stakeholders.

Conclusions:

• The proposed four sets of terms offer a valuable framework for understanding digital evidence in educational assessments.
• This framework can enhance the development and effective use of technology-based assessment systems.
• Clearer data terminology improves the evaluation of learner performance in healthcare settings.