Jove
Visualize
Contact Us

Related Concept Videos

Statistical Significance01:50

Statistical Significance

20.1K
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
20.1K
Confirmation Biases01:31

Confirmation Biases

5.5K
The confirmation bias is the tendency to focus on information that confirms our existing beliefs and ignore information that is inconsistent with our expectations. For example, if you think that your professor is not very nice, you notice all of the instances of rude behavior exhibited by the professor while ignoring the countless pleasant interactions he is involved in on a daily basis. Have you ever fallen prey to the confirmation bias, either as the source or target of such bias?
5.5K
Self-Evaluation: Self-Enhancement and Self-Verification03:00

Self-Evaluation: Self-Enhancement and Self-Verification

5.2K
Social psychologists have documented that feeling good about ourselves and maintaining positive self-esteem is a powerful motivator of human behavior (Tavris & Aronson, 2008). In the United States, members of the predominant culture typically think very highly of themselves and view themselves as good people who are above average on many desirable traits (Ehrlinger, Gilovich, & Ross, 2005). Often, our behavior, attitudes, and beliefs are affected when we experience a threat to our...
5.2K
Cause and Effect01:53

Cause and Effect

10.9K
While variables are sometimes correlated because one does cause the other, it could also be that some other factor, a confounding variable, is actually causing the systematic movement in our variables of interest. For instance, as sales in ice cream increase, so does the overall rate of crime. Is it possible that indulging in your favorite flavor of ice cream could send you on a crime spree? Or, after committing crime do you think you might decide to treat yourself to a cone?
10.9K
Naturalistic Observations02:30

Naturalistic Observations

15.4K
If you want to understand how behavior occurs, one of the best ways to gain information is to simply observe the behavior in its natural context. However, people might change their behavior in unexpected ways if they know they are being observed. How do researchers obtain accurate information when people tend to hide their natural behavior? As an example, imagine that your professor asks everyone in your class to raise their hand if they always wash their hands after using the restroom. Chances...
15.4K
Bias01:22

Bias

4.1K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
4.1K
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies
  1. Home
  2. Subjective Evidence Evaluation Survey For Many-analysts Studies.
  1. Home
  2. Subjective Evidence Evaluation Survey For Many-analysts Studies.

Related Experiment Video

A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences
00:08

A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences

Published on: September 4, 2019

7.0K

Subjective evidence evaluation survey for many-analysts studies.

Alexandra Sarafoglou1, Suzanne Hoogeveen2, Don van den Bergh1

  • 1Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands.

Royal Society Open Science
|July 25, 2024

View abstract on PubMed

Summary
This summary is machine-generated.

Many-analysts studies often use single metrics, limiting insight into analysis choices. A new survey (SEES) offers a nuanced evaluation of methodological appropriateness and evidence strength, enhancing research assessment.

Keywords:
crowdsourcing analysismetascienceopen sciencescientific transparencyteam science

More Related Videos

Electroencephalographic, Heart Rate, and Galvanic Skin Response Assessment for an Advertising Perception Study: Application to Antismoking Public Service Announcements
06:39

Electroencephalographic, Heart Rate, and Galvanic Skin Response Assessment for an Advertising Perception Study: Application to Antismoking Public Service Announcements

Published on: August 28, 2017

14.3K
Use of Galvanic Skin Responses, Salivary Biomarkers, and Self-reports to Assess Undergraduate Student Performance During a Laboratory Exam Activity
07:32

Use of Galvanic Skin Responses, Salivary Biomarkers, and Self-reports to Assess Undergraduate Student Performance During a Laboratory Exam Activity

Published on: February 10, 2016

9.4K

Related Experiment Videos

A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences
00:08

A Cross-Disciplinary and Multi-Modal Experimental Design for Studying Near-Real-Time Authentic Examination Experiences

Published on: September 4, 2019

7.0K
Electroencephalographic, Heart Rate, and Galvanic Skin Response Assessment for an Advertising Perception Study: Application to Antismoking Public Service Announcements
06:39

Electroencephalographic, Heart Rate, and Galvanic Skin Response Assessment for an Advertising Perception Study: Application to Antismoking Public Service Announcements

Published on: August 28, 2017

14.3K
Use of Galvanic Skin Responses, Salivary Biomarkers, and Self-reports to Assess Undergraduate Student Performance During a Laboratory Exam Activity
07:32

Use of Galvanic Skin Responses, Salivary Biomarkers, and Self-reports to Assess Undergraduate Student Performance During a Laboratory Exam Activity

Published on: February 10, 2016

9.4K

Area of Science:

  • Methodology
  • Data Analysis
  • Research Integrity

Background:

  • Many-analysts studies assess empirical claims using multiple analysis teams on the same dataset.
  • Current methods often rely on a single outcome metric (e.g., effect size) per team.
  • This single metric limits a nuanced understanding of how analytical choices impact results.

Purpose of the Study:

  • To develop a more comprehensive method for evaluating evidence in many-analysts studies.
  • To assess the methodological appropriateness and strength of evidence for a hypothesis.
  • To provide a richer understanding of the relationship between analysis choices and outcomes.

Main Methods:

  • Utilized the Delphi consensus technique with 37 experts.
  • Developed an 18-item subjective evidence evaluation survey (SEES).
  • Applied the SEES to pilot data from a prior many-analysts study.
  • Main Results:

    • The SEES provides a subjective evaluation of methodological appropriateness.
    • The survey assesses the strength of evidence for a hypothesis from each analysis team.
    • Pilot data demonstrated the SEES's utility in yielding richer evidence assessments.

    Conclusions:

    • The SEES offers a more nuanced approach to evidence evaluation in many-analysts studies.
    • This method enhances the understanding of how analytical decisions influence research conclusions.
    • The SEES contributes to improving research integrity and the interpretation of empirical claims.