Related Experiment Video
Updated: Dec 3, 2025

Ecotoxicological Methodologies to Evaluate Biomarkers at Different Scales in Neotropical Anurans
Published on: April 28, 2023
Assessing risk of bias in human environmental epidemiology studies using three tools: different conclusions from
Stephanie M Eick1, Dana E Goin1, Nicholas Chartres1
1Program on Reproductive Health and the Environment, Department of Obstetrics, Gynecology and Reproductive Sciences, University of California, San Francisco, USA.
Background:
Systematic reviews are increasingly prevalent in environmental health due to their ability to synthesize evidence while reducing bias. Different systematic review methods have been developed by the US National Toxicology Program's Office of Health Assessment and Translation (OHAT), the US Environmental Protection Agency's (EPA) Integrated Risk Information System (IRIS), and by the US EPA under the Toxic Substances Control Act (TSCA), including the approach to assess risk of bias (ROB), one of the most vital steps which is used to evaluate internal validity of the studies. Our objective was to compare the performance of three tools (OHAT, IRIS, TSCA) in assessing ROB.
Methods:
We selected a systematic review on polybrominated diphenyl ethers and intelligence quotient and/or attention deficit hyperactivity disorder because it had been endorsed by the National Academy of Sciences. Two reviewers followed verbatim instructions from the tools and independently applied each tool to assess ROB in 15 studies previously identified. We documented the time to apply each tool and the impact the ROB ratings for each tool had on the final rating of the quality of the overall body of evidence.
Results:
The time to complete the ROB assessments varied widely (mean = 20, 32, and 40 min per study for the OHAT, IRIS, and TSCA tools, respectively). All studies were rated overall "low" or "uninformative" using IRIS, due to "deficient" or "critically deficient" ratings in one or two domains. Similarly, all studies were rated "unacceptable" using the TSCA tool because of one "unacceptable" rating in a metric related to statistical power. Approximately half of the studies had "low" or "probably low ROB" ratings across all domains with the OHAT and Navigation Guide tools.
Conclusions:
Tools that use overall ROB or study quality ratings, such as IRIS and TSCA, may reduce the available evidence to assess the harms of environmental exposures by erroneously excluding studies, which leads to inaccurate conclusions about the quality of the body of evidence. We recommend using ROB tools that circumvents these issues, such as OHAT and Navigation Guide.
Systematic Review Registration:
This review has not been registered as it is not a systematic review.
More Related Videos
09:33Visualizing Field Data Collection Procedures of Exposure and Biomarker Assessments for the Household Air Pollution Intervention Network Trial in India
Published on: December 23, 2022
08:36Author Spotlight: Evaluating the Adjuvant Efficacy and Safety of Angong Niuhuang Pill in Viral Encephalitis Treatment
Published on: April 19, 2024
Related Concept Videos
Bias in Epidemiological Studies
Strategies for Assessing and Addressing Confounding
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
Types of Biopharmaceutical Studies: Controlled and Non-Controlled Approaches
Non-controlled studies, commonly employed for initial exploration, lack a control group, rendering them susceptible to biases and external influences. In contrast,...
Study Designs in Epidemiology
Observational studies are those where the researcher does not intervene but rather observes natural variations. They include cross-sectional, cohort, and...
Confounding in Epidemiological Studies
Bias
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...