Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Group Design02:01

Group Design

9.0K
The most basic experimental design involves two groups: the experimental group and the control group. The two groups are designed to be the same except for one difference— experimental manipulation. The experimental group gets the experimental manipulation—that is, the treatment or variable being tested—and the control group does not. Since experimental manipulation is the only difference between the experimental and control groups, we can be sure that any differences between...
9.0K
Randomized Experiments01:13

Randomized Experiments

7.1K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
7.1K
Naturalistic Observations02:30

Naturalistic Observations

15.5K
If you want to understand how behavior occurs, one of the best ways to gain information is to simply observe the behavior in its natural context. However, people might change their behavior in unexpected ways if they know they are being observed. How do researchers obtain accurate information when people tend to hide their natural behavior? As an example, imagine that your professor asks everyone in your class to raise their hand if they always wash their hands after using the restroom. Chances...
15.5K
Group Therapy01:26

Group Therapy

67
Group therapy is a sociocultural approach to psychological treatment, where individuals with shared psychological challenges come together under the guidance of a mental health professional. This therapeutic modality offers unique opportunities for individuals to connect, share, and grow within the context of a supportive group. By fostering mutual understanding and collaboration, group therapy can address a range of psychological concerns effectively, often complementing or surpassing the...
67
The Representativeness Heuristic02:13

The Representativeness Heuristic

15.8K
The representative heuristic describes a biased way of thinking, in which you unintentionally stereotype someone or something. For example, you may assume that your professors spend their free time reading books and engaging in intellectual conversation, because the idea of them spending their time playing volleyball or visiting an amusement park does not fit in with your stereotypes of professors.
15.8K
Social Proof00:52

Social Proof

27.7K
Social proof is a form of persuasion based on comparison and conformity. People compare their behavior and actions to what others are doing and will change to conform to do what their peers do.
27.7K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Spatially explicit power analysis reveals challenges for a long-term threatened species monitoring program in Australia.

Ecological applications : a publication of the Ecological Society of America·2026
Same author

AI-assisted teams outperform AI-led teams but not human-only teams in assessing research reproducibility in quantitative social science.

Proceedings of the National Academy of Sciences of the United States of America·2026
Same author

Evaluation of the replicability of systematic reviews with meta-analyses of the effects of health interventions.

Research synthesis methods·2026
Same author

Comparing expert assessments of research quality between the Global North and East Africa.

Philosophical transactions of the Royal Society of London. Series B, Biological sciences·2026
Same author

Reproducibility of meta-analytic results in systematic reviews of interventions: meta-research study.

BMJ medicine·2025
Same author

From policy to practice: progress towards data- and code-sharing in ecology and evolution.

Proceedings. Biological sciences·2025
Same journal

Desert lizards modulate nutritional responses to match seasonal biological needs.

Royal Society open science·2026
Same journal

Multi-generational fidelity, ecological and social determinants of roosting in a cooperatively breeding bird (<i>Argya squamiceps</i>).

Royal Society open science·2025
Same journal

Multifaceted polarization and information reliability in climate change discussions on social media platforms.

Royal Society open science·2025
Same journal

Comparing the kinematics related to inflicted head injury between violent shaking of a 6-week-old and a 1-year-old infant surrogate.

Royal Society open science·2025
Same journal

Partner choice increases observed reciprocity-based cooperation but decreases unobserved stake-based cooperation.

Royal Society open science·2025
Same journal

Importation models for travel-related SARS-CoV-2 cases reported in Newfoundland and Labrador during the COVID-19 pandemic.

Royal Society open science·2025
See all related articles

Related Experiment Video

Updated: Jul 27, 2025

The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups
14:14

The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups

Published on: May 13, 2022

5.9K

Predicting and reasoning about replicability using structured groups.

Bonnie C Wintle1, Eden T Smith2, Martin Bush2

  • 1MetaMelb Research Initiative, School of Ecosystem and Forest Sciences, University of Melbourne, Parkville 3010, Australia.

Royal Society Open Science
|June 9, 2023
PubMed
Summary
This summary is machine-generated.

Experts accurately predicted research replicability using the IDEA protocol. Broader reasoning, considering factors like effect size and reputation, improved prediction accuracy in social and behavioral sciences.

Keywords:
expert judgementforecastingmeta-researchmetasciencemixed methodsreplication

More Related Videos

The Replica Set Method: A High-throughput Approach to Quantitatively Measure Caenorhabditis elegans Lifespan
11:58

The Replica Set Method: A High-throughput Approach to Quantitatively Measure Caenorhabditis elegans Lifespan

Published on: June 29, 2018

9.5K
Heuristic Mining of Hierarchical Genotypes and Accessory Genome Loci in Bacterial Populations
08:03

Heuristic Mining of Hierarchical Genotypes and Accessory Genome Loci in Bacterial Populations

Published on: December 7, 2021

2.2K

Related Experiment Videos

Last Updated: Jul 27, 2025

The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups
14:14

The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups

Published on: May 13, 2022

5.9K
The Replica Set Method: A High-throughput Approach to Quantitatively Measure Caenorhabditis elegans Lifespan
11:58

The Replica Set Method: A High-throughput Approach to Quantitatively Measure Caenorhabditis elegans Lifespan

Published on: June 29, 2018

9.5K
Heuristic Mining of Hierarchical Genotypes and Accessory Genome Loci in Bacterial Populations
08:03

Heuristic Mining of Hierarchical Genotypes and Accessory Genome Loci in Bacterial Populations

Published on: December 7, 2021

2.2K

Area of Science:

  • Social and behavioral sciences
  • Research methodology
  • Scientific judgment

Background:

  • Assessing the replicability of research is crucial for scientific progress.
  • Judgments about replicability can be influenced by various cognitive and contextual factors.

Purpose of the Study:

  • To explore how experts judge research replicability.
  • To identify factors driving accurate replicability judgments.

Main Methods:

  • Utilized a mixed-methods approach with the IDEA protocol (Investigate, Discuss, Estimate, Aggregate).
  • Employed five expert groups to evaluate 25 research claims with replication data.
  • Collected qualitative and quantitative data on participants' judgments and reasoning.

Main Results:

  • Participants achieved 84% accuracy in predicting research replicability.
  • Broader reasoning and consideration of factors like 'effect size' and 'reputation' correlated with higher accuracy.
  • Evidence suggests a link between statistical literacy and accuracy in replicability judgments.

Conclusions:

  • Expert judgments can be highly accurate in predicting research replicability.
  • Encouraging diverse reasoning strategies and statistical literacy may enhance the reliability of scientific assessments.