Assessing the effectiveness of metaHelper: a randomized controlled trial of a web application for statistical...

To test whether the use of a web application (www.metaHelper.eu) can reduce errors and time needed for statistical transformations commonly performed for meta-analysis.

We developed an R package and a corresponding web application to streamline statistical transformations in meta-analysis. A parallel randomized controlled trial following the Consolidated Standards of Reporting Trials guidelines was conducted. The trial was preregistered, and ethical approval was obtained from the University of Krems' ethics committee. We conducted a parallel randomized controlled trial with 26 participants, including researchers and students with varying levels of statistical expertise. Participants performed six statistical calculations commonly used in meta-analysis: computing pooled standard deviations, converting standard errors, and calculating standardized mean differences for both 2-group and 3-arm studies. Performance was evaluated based on accuracy and time taken to complete each task. A Bayesian mixed-effects model was used to analyze the results. Participants were randomly assigned to either the experimental group (metaHelper) or the control group (allowed to use internet sources, the Cochrane Handbook, or any other sources chosen by the participant).

Main outcomes were correctness of calculations and time taken to complete tasks. The metaHelper group had a higher probability of providing correct answers (85%) compared to the control group (31%). Additionally, the metaHelper group completed an average of 133 seconds per task faster (95% credibility interval: 83-180).

MetaHelper effectively enhances the accuracy and efficiency of statistical transformations required for meta-analysis.

Meta-analysis, a method for combining research results, often requires researchers to transform statistical data, like converting standard errors to standard deviations. These transformations are critical can be prone to errors, especially when done manually or with limited guidance. To address this, we developed metaHelper (www.metahelper.eu), a user-friendly web application and R package designed to assist researchers with these calculations. In our study, we conducted a randomized controlled trial to compare metaHelper with traditional methods like handbook instructions and freely available online tools (whatever our participants wanted to use). Twenty-six researchers and students participated, performing 6 common statistical calculations. We measured their accuracy and the time they took to complete each task. Results statistical model: Probability of achieving a correct answer. Results statistical model: Average time needed per task for both groups.