Limited consensus in expert opinions on studies evaluating the design, conduct, analysis, or reporting of health...

Methodological studies critically evaluate how health research is designed, conducted, analyzed, and reported. Despite their growing importance, currently, there is no reporting tailored to this type of research, which hampers the visibility, reproducibility, and overall utility of the findings of methodological studies.

We administered a survey to researchers with expertise in designing and performing methodological studies to gather their opinions on appropriate terminology, how they should be categorized, and key reporting elements. Quantitative data were analyzed descriptively, with a content validity ratio applied to determine appropriateness. Qualitative survey responses were analyzed using inductive content analysis.

Of 499 invited, a total of 119 participants completed the survey (response rate 23%). None of the 13 proposed nomenclatures met the threshold for appropriateness. Of the four proposed study categories, two (study aim and study design) were retained based on expert ratings. Among 23 proposed reporting items, 15 were endorsed for further evaluation. Qualitative responses were condensed and categorized, identifying the importance of flexibility in terminology and concerns about categorization.

There is substantial disagreement among experts regarding key aspects of methodological studies, particularly related to terminology. While some agreement was observed around study categories and reporting elements, diverse and sometimes conflicting perspectives underscore the complexity of standardizing methodological studies. These findings reinforce the need for a collaborative consensus process to develop reporting guidance that is both practical and adaptable to the nuances of this field.