Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Stereotype Content Model02:16

Stereotype Content Model

14.0K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.0K
Bias01:22

Bias

3.7K
Bias refers to any tendency that prevents a question from being considered unprejudiced. In research, bias occurs when one outcome or answer is selected or encouraged over others in sampling or testing. Bias can occur during any research phase, including study design, data collection, analysis, and publication.
In statistics, a sampling bias is created when a sample is collected from a population, and some members of the population are not as likely to be chosen as others (remember, each member...
3.7K
In- and Out-Groups01:31

In- and Out-Groups

38.9K
People all belong to a gender, race, age, and social economic group. These groups provide a powerful source of our identity and self-esteem (Tajfel & Turner, 1979) and serve as our in-groups. An in-group is a group that we identify with or see ourselves as belonging to.
38.9K
Strategies for Assessing and Addressing Confounding01:25

Strategies for Assessing and Addressing Confounding

83
Confounding is a critical issue in epidemiological studies, often leading to misleading conclusions about associations between exposures and outcomes. It occurs when the relationship between the exposure and the outcome is mixed with the effects of other factors that influence the outcome. Given that, addressing confounding is of high importance for drawing accurate inferences in research.
Confounding can be addressed at both the design phase of a study and through analytical methods after data...
83
Improving Translational Accuracy02:07

Improving Translational Accuracy

9.1K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
9.1K
Stereotypes, Prejudice, and Discrimination02:55

Stereotypes, Prejudice, and Discrimination

90.0K
Humans are very diverse and although we share many similarities, we also have many differences. The social groups we belong to help form our identities (Tajfel, 1974). These differences may be difficult for some people to reconcile, which may lead to prejudice toward people who are different. Prejudice is a negative attitude and feeling toward an individual based solely on one’s membership in a particular social group (Allport, 1954; Brown, 2010). Prejudice is common against people who...
90.0K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Comparative Study of Molecular Descriptors and AI-Based Embeddings for Toxicity Prediction.

Chemical research in toxicology·2025
Same author

Leveraging FDA Labeling Documents and Large Language Model to Enhance Annotation, Profiling, and Classification of Drug Adverse Events with AskFDALabel.

Drug safety·2025
Same author

PERform: assessing model performance with predictivity and explainability readiness formula.

Journal of environmental science and health. Part C, Toxicology and carcinogenesis·2024
Same author

RxBERT: Enhancing drug labeling text mining and analysis with AI language modeling.

Experimental biology and medicine (Maywood, N.J.)·2024
Same author

Measurement and Mitigation of Bias in Artificial Intelligence: A Narrative Literature Review for Regulatory Science.

Clinical pharmacology and therapeutics·2023
Same author

A Weakly Supervised Deep Learning Framework for Whole Slide Classification to Facilitate Digital Pathology in Animal Study.

Chemical research in toxicology·2023

Related Experiment Video

Updated: Jun 7, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

504

Enhancing Bias Assessment for Complex Term Groups in Language Embedding Models: Quantitative Comparison of Methods.

Magnus Gray1, Mariofanna Milanova2, Leihong Wu1

  • 1Division of Bioinformatics & Biostatistics, National Center for Toxicological Research, US Food and Drug Administration, Jefferson, AR, United States.

JMIR Medical Informatics
|November 15, 2024
PubMed
Summary

A new method, the standard deviation-word embedding association test (SD-WEAT), offers a more robust and reliable way to measure bias in artificial intelligence (AI) language models. This improved technique addresses limitations of previous methods, ensuring fairer AI development.

Keywords:
AIAI language modelsAI-powered toolNLPapplicationartificial intelligenceassessmentbiasbias measurementdecision-makinginput embeddingslanguage modelsnatural language processing

More Related Videos

Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
06:48

Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment

Published on: June 25, 2019

9.1K
A Semantic Priming Event-related Potential ERP Task to Study Lexico-semantic and Visuo-semantic Processing in Autism Spectrum Disorder
08:17

A Semantic Priming Event-related Potential ERP Task to Study Lexico-semantic and Visuo-semantic Processing in Autism Spectrum Disorder

Published on: April 12, 2018

10.5K

Related Experiment Videos

Last Updated: Jun 7, 2025

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

504
Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment
06:48

Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment

Published on: June 25, 2019

9.1K
A Semantic Priming Event-related Potential ERP Task to Study Lexico-semantic and Visuo-semantic Processing in Autism Spectrum Disorder
08:17

A Semantic Priming Event-related Potential ERP Task to Study Lexico-semantic and Visuo-semantic Processing in Autism Spectrum Disorder

Published on: April 12, 2018

10.5K

Area of Science:

  • Natural Language Processing (NLP)
  • Artificial Intelligence (AI) Ethics
  • Machine Learning Model Evaluation

Background:

  • Artificial intelligence (AI) systems are increasingly used across industries, but they can perpetuate and amplify societal biases.
  • Existing methods for measuring bias in AI, such as the word embedding association test (WEAT), have limitations including non-robustness and reliance on predefined word groups.
  • There is a critical need for improved methods to accurately measure and mitigate bias in AI to ensure fair and ethical development.

Purpose of the Study:

  • To introduce a modified and more robust measure for detecting bias in AI language models.
  • To address the limitations of the traditional word embedding association test (WEAT).
  • To enhance the applicability and reliability of bias measurement in natural language processing embeddings.

Main Methods:

  • Introduction of the standard deviation-word embedding association test (SD-WEAT), a novel modification of the WEAT.
  • The SD-WEAT calculates bias by analyzing the standard deviation of multiple WEAT permutations.
  • Evaluation of bias and stability in prominent language embedding models: GloVe, Word2Vec, and BERT.

Main Results:

  • The SD-WEAT demonstrated comparable results to the WEAT, with high correlations in bias scores (r=0.786) and P values (r=0.776).
  • The SD-WEAT overcomes WEAT limitations by removing the need for predefined attribute groups and reducing outlier impact through multiple runs.
  • The new method proved to be more consistent and reliable than the original WEAT for bias assessment.

Conclusions:

  • The SD-WEAT presents a promising advancement for the robust measurement of bias in AI language model embeddings.
  • This method offers greater accessibility and reliability for evaluating and mitigating bias in AI.
  • The SD-WEAT contributes to the development of fairer and more equitable AI technologies.