Comparison of Study Quality as Determined by Standard Research and Community Engagement Metrics: A Pilot Study on...

The purpose of this review is to compare research evaluation tools to determine whether the tools typically used for assessing the quality of research adequately address issues of Indigenous health and culture, particularly when the studies are intended to benefit Indigenous peoples in urban, regional, rural, and remote settings. Our previously published systematic review evaluated studies about breast cancer using a modified Indigenous community engagement tool (CET). In this study, we evaluated the same studies using two commonly used tools: the Critical Appraisal Skills Programme (CASP) for qualitative research; and the Effective Public Health Practice Project (EPHPP) for quantitative research. The results were then compared to ascertain whether there was alignment between performances in terms of engagement and the CASP/EPHPP metrics. Of the 15 papers, 3 papers scored weakly on both metrics, and are therefore the least likely to offer reliable findings, while 2 papers scored strongly on both metrics, and are therefore the most likely to offer reliable findings. Beyond this summation, it was clear that the results did not align and, therefore, could not be used interchangeably when applied to research findings intended to benefit Indigenous peoples. There does not appear to be a pattern in the relationship between the reliability of the studies and the study settings. In order to address disparities in health outcomes, we must assess research through a typical research quality and cultural engagement and settings lens, ensuring that there is rigour in all aspects of the studies.