Test Security and the Pandemic: Comparison of Test Center and Online Proctor Delivery Modalities

Area of Science:

• Psychometrics
• Educational Measurement
• Testing Security

Background:

• Limited published data exists on testing program security, especially comparing in-person (TC) and remote online proctored (OP) modalities.
• Understanding the relative security of different testing formats is crucial for maintaining assessment integrity.

Purpose of the Study:

• To examine indicators of test security violations across diverse testing programs.
• To compare the security of in-person (TC) versus remote online proctored (OP) testing modalities.
• To analyze changes in security indicators following the introduction of OP testing within a large program.

Main Methods:

• Analysis of response overlap as an indicator of collusion.
• Scrutiny of various test security indicators (e.g., response times, score gains, differential functioning) over 14 months.
• Comparison of security metrics before and after implementing OP testing, including test-retest modality analysis.

Main Results:

• High response overlap was analyzed as a potential indicator of cheating across different testing modalities and center types.
• Introduction of OP testing was examined alongside indicators like fast/slow correct responses and score gains.
• Differential person functioning and test-retest score gains were assessed across modalities, including modality switching.

Conclusions:

• The introduction of remote online proctored (OP) testing may present new challenges to test security.
• Further research is needed to fully understand and mitigate potential security risks associated with OP testing compared to traditional in-person testing.