Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Significance Testing: Overview01:04

Significance Testing: Overview

12.9K
Significance testing is a set of statistical methods used to test whether a claim about a parameter is valid. In analytical chemistry, significance testing is used primarily to determine whether the difference between two values comes from determinate or random errors. The effect of a particular change in the measurement protocol, analyst, or sample itself can cause a deviation from the expected result. In the case of a suspected deviation/outlier, we need to be able to confirm mathematically...
12.9K
Statistical Significance01:37

Statistical Significance

23.3K
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
23.3K
One-Way ANOVA: Unequal Sample Sizes01:15

One-Way ANOVA: Unequal Sample Sizes

6.9K
One-way ANOVA can be performed on three or more samples of unequal sizes. However, calculations get complicated when sample sizes are not always the same. So, while performing ANOVA with unequal samples size, the following equation is used:
6.9K
Identifying Statistically Significant Differences: The F-Test01:14

Identifying Statistically Significant Differences: The F-Test

4.0K
The F-test is used to compare two sample variances to each other or compare the sample variance to the population variance. It is used to decide whether an indeterminate error can explain the difference in their values. The underlying assumptions that allow the use of the F-test include the data set or sets are normally distributed, and the data sets are independent of each other. The test statistic F is calculated by dividing one variance by another. In other words, the square of one standard...
4.0K
Statistical Methods to Analyze Parametric Data: ANOVA01:12

Statistical Methods to Analyze Parametric Data: ANOVA

1.9K
Analysis of Variance, or ANOVA, is a powerful statistical technique used to analyze parametric data, primarily in research and experimental studies. It's designed to compare the means of two or more groups, assisting researchers in identifying any significant differences between these group means. There are two main types of ANOVA based on the complexity of the analysis: one-way and two-way.
One-way ANOVA is applied when a single independent variable or factor is scrutinized. It compares...
1.9K
One-Way ANOVA: Equal Sample Sizes01:15

One-Way ANOVA: Equal Sample Sizes

4.3K
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
4.3K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

The perceptual span in dyslexic reading and visual search.

Dyslexia (Chichester, England)·2024
Same author

Dyslexics Exhibit an Orthographic, Not a Phonological Deficit in Lexical Decision.

Language, cognition and neuroscience·2024
Same author

The effect of sudden-onset distractors on reading efficiency and comprehension.

Quarterly journal of experimental psychology (2006)·2022
Same author

Spousal emotional support and relationship quality buffers pupillary response to horror movies.

PloS one·2021
Same author

Best practices in eye tracking research.

International journal of psychophysiology : official journal of the International Organization of Psychophysiology·2020
Same author

The effect of convolving word length, word frequency, function word predictability and first pass reading time in the analysis of a fixation-related fMRI dataset.

Data in brief·2019

Related Experiment Video

Updated: Mar 15, 2026

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
04:35

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach

Published on: July 3, 2020

3.8K

Evaluating significance in linear mixed-effects models in R.

Steven G Luke1

  • 1Department of Psychology, Brigham Young University, 1001 Spencer W. Kimball Tower, Provo, UT, 84602, USA. steven_luke@byu.edu.

Behavior Research Methods
|September 14, 2016
PubMed
Summary

For mixed-effects models, common p-value methods like likelihood ratio tests and Wald t-tests (t-as-z) are anti-conservative. Kenward-Roger and Satterthwaite approximations offer better Type 1 error control, especially with REML fitting.

Keywords:
Linear mixed-effects modelsStatisticsType 1 errorlme4p-values

More Related Videos

Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
06:48

Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment

Published on: June 25, 2019

9.9K
The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups
14:14

The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups

Published on: May 13, 2022

6.4K

Related Experiment Videos

Last Updated: Mar 15, 2026

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
04:35

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach

Published on: July 3, 2020

3.8K
Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
06:48

Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment

Published on: June 25, 2019

9.9K
The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups
14:14

The Innovation Arena: A Method for Comparing Innovative Problem-Solving Across Groups

Published on: May 13, 2022

6.4K

Area of Science:

  • Statistics
  • Computational Biology
  • Data Science

Background:

  • Mixed-effects models are increasingly used in experimental data analysis.
  • The lme4 package in R lacks clear standards for evaluating fixed effect significance (p-values).
  • Researchers often require p-values for reporting model significance.

Purpose of the Study:

  • To evaluate common methods for determining p-values in mixed-effects models.
  • To identify reliable methods for significance testing in mixed-effects models, especially with smaller sample sizes.

Main Methods:

  • Simulations were conducted to assess Type 1 error rates.
  • Evaluated methods included likelihood ratio tests, t-as-z approximation, parametric bootstrapping, and Kenward-Roger/Satterthwaite approximations.
  • Models were fitted using REML (Restricted Maximum Likelihood).

Main Results:

  • Likelihood ratio tests and t-as-z methods were found to be anti-conservative, particularly with smaller sample sizes.
  • Parametric bootstrapping, Kenward-Roger, and Satterthwaite approximations were evaluated.
  • Kenward-Roger and Satterthwaite approximations yielded Type 1 error rates closest to the desired 0.05 level.

Conclusions:

  • When fitting mixed-effects models using REML, the Kenward-Roger and Satterthwaite approximations provide more accurate p-values.
  • These approximations maintain acceptable Type 1 error rates even for smaller sample sizes, offering a more reliable approach to significance testing.