Related Experiment Video
Updated: Jul 17, 2026
Evaluating Tests of Cognition using a Computerized Touch-Sensitive Tablet, Eye Tracking, and Functional Magnetic Resonance Imaging
Published on: January 30, 2026
Cognitive impact assessment of electronic knowledge resources: a mixed methods evaluation study of a handheld
1Department of Family Medicine, McGill University, 517 Pine Avenue West, Montreal, Quebec, Canada, H2W 1S4.
Rationale:
We recently proposed a new method to systematically assess the cognitive impact of knowledge resources on health professionals.
Objective:
To describe promises and shortcomings of a handheld computer prototype of this method.
Background:
We developed an impact scale, and combined this scale with a Computerized Ecological Momentary Assessment technique.
Method:
We conducted a mixed methods evaluation study using a 7-item scale within a questionnaire linked to a commercial knowledge resource. Over two months of Family Medicine training, 17 residents assessed the impact of 1,981 information hits retrieved on handheld computer. From observations, log-reports, archives of hits and interviews, we examined issues associated with hardware, software and the questionnaire.
Findings:
Fifteen residents found the questionnaire clearly written, and only one pointed to the questionnaire as a major reason for their low level of use of the resource. Residents reported technical problems (e.g. screen trouble) or limitations (e.g. limited tracking function) and socio-technical issues (e.g. software dependency).
Conclusion:
Lessons from this study suggest improvements to guide future implementation of our method for assessing the cognitive impact of knowledge resources on health professionals.
