The Revised METRIQ Score: A Quality Evaluation Tool for Online Educational Resources

Area of Science:

• Medical Education
• Digital Health
• Quality Assessment

Background:

• Online medical education resources are rapidly expanding, necessitating robust quality evaluation methods.
• The Medical Education Translational Resources: Impact and Quality (METRIQ) study initially developed the METRIQ-8 instrument to assess blog quality.
• This study focused on gathering user feedback to refine the METRIQ-8 instrument.

Purpose of the Study:

• To evaluate the usability and clarity of the METRIQ-8 quality assessment instrument for medical blogs.
• To collect user feedback for the iterative development of an improved quality assessment tool.
• To inform the creation of the revised METRIQ (rMETRIQ) score.

Main Methods:

• Participants (309 emergency medicine professionals and trainees) used the METRIQ-8 to rate five clinical emergency medicine blog posts.
• User feedback on the METRIQ-8 was collected via a 7-point Likert scale and free-text comments, focusing on ease of use and clarity.
• Descriptive statistics and thematic analysis of comments were performed to identify areas for improvement.

Main Results:

• Most participants found the METRIQ-8 easy to use and likely to recommend.
• Thematic analysis indicated a need for clearer questions, a shorter scale, defined anchors, and removal of an "unsure" option.
• These findings directly informed the development of the revised METRIQ (rMETRIQ) score.

Conclusions:

• User feedback on the METRIQ-8 instrument was instrumental in developing the rMETRIQ score.
• The rMETRIQ score demonstrates improved clarity and usability compared to its predecessor.
• Further research is needed to establish the validity of the rMETRIQ score.