Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Data Validation01:15

Data Validation

542
Method validation is a crucial process in analytical chemistry designed to confirm that a given method consistently produces reliable and high-quality results. This process is essential when a method is applied to different sample matrices or when procedural modifications are made, ensuring that the results meet acceptable standards across various applications.
Key parameters for method validation include:
542
Uncertainty in Measurement: Accuracy and Precision03:37

Uncertainty in Measurement: Accuracy and Precision

99.2K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value. 
99.2K
Systematic Error: Methodological and Sampling Errors01:15

Systematic Error: Methodological and Sampling Errors

8.4K
In the case of systematic errors, the sources can be identified, and the errors can be subsequently minimized by addressing these sources. According to the source, systematic errors can be divided into sampling, instrumental, methodological, and personal errors.
Sampling errors originate from improper sampling methods or the wrong sample population. These errors can be minimized by refining the sampling strategy. Defective instruments or faulty calibrations are the sources of instrumental...
8.4K
Random and Systematic Errors01:20

Random and Systematic Errors

14.3K
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...
14.3K
Reliability and Validity01:29

Reliability and Validity

13.7K
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
13.7K
Accuracy and Errors in Hypothesis Testing01:13

Accuracy and Errors in Hypothesis Testing

545
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
545

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Safety Listening in High-Risk Situations: A Qualitative Analysis of Responses to Safety Voice in Aviation.

Risk analysis : an official publication of the Society for Risk Analysis·2025
Same author

Conversational repairs on Reddit: Widely initiated but often uncompleted.

PloS one·2025
Same author

Online patient feedback as a safety valve: An automated language analysis of unnoticed and unresolved safety incidents.

Risk analysis : an official publication of the Society for Risk Analysis·2022
Same author

Do national policies for complaint handling in English hospitals support quality improvement? Lessons from a case study.

Journal of the Royal Society of Medicine·2022
Same author

Identifying and encouraging high-quality healthcare: an analysis of the content and aims of patient letters of compliment.

BMJ quality & safety·2020
Same author

Learning from complaints in healthcare: a realist review of academic literature, policy evidence and front-line insights.

BMJ quality & safety·2020
Same journal

Bayesian evaluation for latent variable models: A tutorial on computing information criteria and bayes factors with the r package bleval.

Psychological methods·2026
Same journal

A stochastic block prior for clustering in graphical models.

Psychological methods·2026
Same journal

Three-level vector autoregressive models.

Psychological methods·2026
Same journal

Scaling cognitive modeling to big data: A deep learning approach to studying individual differences in evidence accumulation model parameters.

Psychological methods·2026
Same journal

Best practices in multilevel modeling for within-cluster group comparisons: An evaluation of coding strategies reflecting group composition and heterogeneity.

Psychological methods·2026
Same journal

A unified framework for psychometrics in experimental psychology: The standardized generalized hierarchical factor model.

Psychological methods·2026
See all related articles

Related Experiment Video

Updated: Jan 6, 2026

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

8.0K

The repeated adjustment of measurement protocols method for developing high-validity text classifiers.

Alex Goddard1, Alex Gillespie1

  • 1Department of Psychological and Behavioural Science, London School of Economics and Political Science.

Psychological Methods
|October 6, 2025
PubMed
Summary
This summary is machine-generated.

Developing high-validity text classifiers in psychology requires integrating manual coding and computational algorithms. The Repeated Adjustment of Measurement Protocols (RAMP) method addresses this by iteratively refining concepts and constructs for improved accuracy.

More Related Videos

Design and Evaluation of Smart Glasses for Food Intake and Physical Activity Classification
07:47

Design and Evaluation of Smart Glasses for Food Intake and Physical Activity Classification

Published on: February 14, 2018

11.8K
Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates
08:56

Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates

Published on: January 13, 2023

2.8K

Related Experiment Videos

Last Updated: Jan 6, 2026

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

8.0K
Design and Evaluation of Smart Glasses for Food Intake and Physical Activity Classification
07:47

Design and Evaluation of Smart Glasses for Food Intake and Physical Activity Classification

Published on: February 14, 2018

11.8K
Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates
08:56

Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates

Published on: January 13, 2023

2.8K

Area of Science:

  • Psychology
  • Computational Linguistics
  • Data Science

Background:

  • Developing text classifiers in psychology relies on manual coding, but its evaluation is often separate from algorithm evaluation.
  • This separation hinders the iterative process of identifying and resolving conceptual and measurement issues crucial for high-validity classifiers.

Purpose of the Study:

  • Introduce the Repeated Adjustment of Measurement Protocols (RAMP) method for developing high-validity text classifiers in psychology.
  • Integrate best practices from content analysis, data science, and psychology to create a robust classifier development framework.

Main Methods:

  • The RAMP method comprises three stages: manual coding, classifier development, and integrative evaluation.
  • It utilizes an inference loop for iterative refinement of concepts and constructs based on empirical data.
  • A case study involved manual coding of 21,815 sentences and developing rule-based, supervised machine learning, and large language model classifiers.

Main Results:

  • Manual coding achieved high intersubjective agreement (Krippendorff's α = .79).
  • Supervised machine learning (Bidirectional Encoder Representations From Transformers) achieved the highest classifier performance (Matthews correlation coefficient [MCC] = 0.69).
  • The RAMP method successfully identified and addressed a concept validity issue related to misunderstandings.

Conclusions:

  • RAMP operationalizes validity as a dynamic process of iterative adjustment toward intersubjective agreement.
  • Integrating manual coding and classifier development is essential for addressing concept validity problems.
  • The RAMP method offers a structured approach to enhance the validity of text classifiers in psychological research.