Related Experiment Video
Updated: Jun 5, 2025
07:35
Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
Published on: October 11, 2018
7.4K
Exploring the Impact of Deleting (or Retaining) a Biased Item: A Procedure Based on Classification Accuracy
1University of Southern California, Los Angeles, USA.
Assessment
|December 10, 2024
Summary
This study introduces item-level effect size indices to assess the impact of item deletion on psychological test fairness and performance. The new methods and the R package unbiasr aid informed decision-making in test development and application.
Area of Science:
- Psychometrics
- Psychological Measurement
- Statistical Modeling
Background:
- Psychological tests are crucial for individual classification in high-stakes settings.
- Measurement invariance is essential for valid group comparisons, but rarely achieved in practice.
- Existing classification accuracy frameworks may not fully address item-level bias in classification.
Purpose of the Study:
- To develop item-level effect size indices for quantifying the impact of item deletion on test fairness and performance.
- To provide practical guidance for test developers and users when dealing with measurement noninvariance.
- To introduce the R package 'unbiasr' for implementing these new methods.
Main Methods:
- Development of item-level effect size indices.
- Quantification of the impact of item deletion/retention on test performance and fairness.
- Illustrative example application of the proposed indices.
- Implementation of methods in the R package 'unbiasr'.
Main Results:
- Proposed indices allow for informed decisions regarding item retention or deletion.
- Quantifies the trade-off between improving fairness and maintaining test performance.
- Demonstrates the practical utility of the 'unbiasr' package.
Conclusions:
- Item-level analysis provides a more nuanced approach to addressing measurement noninvariance in psychological testing.
- The developed indices and 'unbiasr' package offer valuable tools for enhancing fairness and validity in test classification.
- Informed decisions about item management can optimize both test fairness and performance.
More Related Videos
12:18A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
Published on: January 11, 2020
7.5K
08:05A Prediction Error-driven Retrieval Procedure for Destabilizing and Rewriting Maladaptive Reward Memories in Hazardous Drinkers
Published on: January 5, 2018
9.7K
Related Concept Videos
01:22Bias
3.7K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
3.7K
01:31Confirmation Biases
5.4K
The confirmation bias is the tendency to focus on information that confirms our existing beliefs and ignore information that is inconsistent with our expectations. For example, if you think that your professor is not very nice, you notice all of the instances of rude behavior exhibited by the professor while ignoring the countless pleasant interactions he is involved in on a daily basis. Have you ever fallen prey to the confirmation bias, either as the source or target of such bias?
5.4K
01:13Accuracy and Errors in Hypothesis Testing
175
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
175
01:29Bias in Epidemiological Studies
155
Biases can arise at various stages of research, from study design and data collection to analysis and interpretation. Recognizing and addressing these biases is essential to ensure the validity and reliability of epidemiological findings.Broadly speaking, biases in epidemiology fall into three main categories: selection bias, information bias, and confounding. A more detailed description of possible biases is:
155
02:16Stereotype Content Model
14.0K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.0K
01:00Detection of Gross Error: The Q Test
5.6K
When one or more data points appear far from the rest of the data, there is a need to determine whether they are outliers and whether they should be eliminated from the data set to ensure an accurate representation of the measured value. In many cases, outliers arise from gross errors (or human errors) and do not accurately reflect the underlying phenomenon. In some cases, however, these apparent outliers reflect true phenomenological differences. In these cases, we can use statistical methods...
5.6K