Related Experiment Video
Updated: Jul 12, 2026

Using Visual and Narrative Methods to Achieve Fair Process in Clinical Care
Published on: February 16, 2011
The importance of adequate user feedback for developing appraisal tools for evidence synthesis-the case of TACIT
Andreas Lundh1, Isabelle Boutron2, Lesley A Stewart3
1Cochrane Denmark and Centre for Evidence-Based Medicine Odense, Department of Clinical Research, University of Southern Denmark, Odense, Denmark; Open Patient data Explorative Network (OPEN), Odense University Hospital, Odense, Denmark; Department of Respiratory Medicine and Infectious Diseases, Copenhagen University Hospital - Bispebjerg and Frederiksberg, Copenhagen, Denmark.
Background And Objective:
The validity of randomized trials included in evidence syntheses is typically assessed using appraisal tools. Such tools are often developed by a group of experts and usually involve some feedback from intended tool users.
Study Design And Setting:
In this commentary, we provide an overview of user testing procedures in the development of four selected appraisal tools for evidence synthesis of randomized trials and describe and discuss the user testing procedures in TACIT, a new Tool for Addressing Conflicts of Interest in Trials (TACIT) that we have developed.
Results:
The procedures for user feedback varied between the selected appraisal tools in relation to the type of users and trials involved, in which setting user testing was done, and how feedback was collected. In TACIT, informal user feedback was provided in the early phase of tool development and formalized feedback was provided to a late draft of the tool. The formalized feedback involved workshop participants using the tool on a preselected set of trials and systematic reviewers using the tool in their own review. Feedback formats included both semistructured interviews and a questionnaire.
Conclusions:
Adequate user feedback is important for the development of appraisal tools, and we suggest striving for an iterative process of tool revisions informed by different groups of intended users applying the tool in different contexts and providing feedback using diverse formats. Feedback from users after public release of the tool, but before final publication, has the potential to reach a large number of users applying the tool on a broad set of trials. We encourage systematic reviewers to apply the publicly released version of TACIT and contact us with feedback.
Related Concept Videos
Ethics in Research
Decision Making: P-value Method
First, a specific claim about the population parameter is proposed. The claim is based on the research question and is stated in a simple form. Further, an opposing statement to the claim is also stated. These statements can act as null and alternative hypotheses: a null hypothesis would be a neutral statement while the alternative hypothesis can have a...
Bias
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
Strategies for Assessing and Addressing Confounding
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
Lazarus's Cognitive Appraisal Theory
Primary Appraisal:...
Halo Effect
