An exploratory study into the effect of time-restricted internet access on face-validity, construct validity and...

Area of Science:

• Medical Education
• Assessment and Evaluation

Background:

• Traditional postgraduate medical education relies on yearly formative knowledge testing (progress testing).
• Existing progress tests demonstrate limitations in construct-validity and reliability.
• Enhancing test authenticity is a key strategy to improve validity and reliability.

Purpose of the Study:

• To investigate if limited internet access during progress tests improves perceived authenticity (face-validity).
• To assess the impact of internet access on the construct-validity and reliability of postgraduate progress testing.

Main Methods:

• Postgraduate trainees participated in a study involving a traditional pen-and-paper test with 30 minutes of internet access at the end.
• A questionnaire assessed trainees' perception of test face-validity before and after the internet-enabled test.

Main Results:

• Significant increases in mean test scores were observed across all training years with internet access.
• Trainees reported improved face-validity and a desire for continued internet access in future tests.
• Internet access did not lead to improvements in the test's construct-validity or reliability.

Conclusions:

• Incorporating limited internet access enhances the face-validity and performance of postgraduate progress tests.
• Despite positive impacts on face-validity and performance, internet access did not alter the test's reliability or construct-validity.