The Development of a Checklist to Enhance Methodological Quality in Intervention Programs

Area of Science:

• Research Methodology
• Evidence Synthesis
• Biostatistics

Background:

• Assessing the methodological quality of primary studies is vital for reliable meta-analyses and systematic reviews.
• Current methods for evaluating quality lack clear criteria, leading to inconsistencies and debates regarding representativeness, utility, and feasibility.

Purpose of the Study:

• To systematize and summarize existing literature on methodological quality in primary studies.
• To propose a concise, 12-item checklist for empirically defining the methodological quality of primary studies.
• To present an inter-coder reliability study for the developed checklist.

Main Methods:

• A content validity study was conducted to develop the 12-item checklist.
• An inter-coder reliability study was performed to assess the consistency of the checklist's application.
• The development process emphasized clearly specified criteria for item inclusion.

Main Results:

• A 12-item checklist for assessing methodological quality of primary studies was developed.
• The checklist demonstrated substantial inter-coder agreement across its items.
• The checklist serves as an assessment tool addressing representativeness, utility, and feasibility of quality items.

Conclusions:

• The proposed checklist provides practitioners and researchers with clear criteria for evaluating methodological quality.
• Individual methodological features should be considered indicators of quality in intervention programs.
• Enhancing methodological quality improves intervention programs and cumulative knowledge from meta-analyses.