Using qualitative data for estimating construct validity of standardized measures
Related Experiment Video
Updated: Nov 3, 2026

Qualitative and Quantitative Validation of Tools with Rating Scales Aimed at Assessing the Quality of University Service-Learning
Published on: August 29, 2025
K J Aroian1, N Schappler-Morris
1Boston College School of Nursing, USA.
Abstract:
This paper describes efforts to develop a method of using qualitative interview data for investigating the construct validity of standardized measures. Illustration is provided through a multimethod validation study that examined the concurrence between observer assessments of interview data and a standardized paper and pencil measure-the Demands of Immigration Scale (DI). Specific aims of the multimethod validation study included (1) determining interrater reliability of the Demands of Immigration Observer Rating Scale (DI-ORS), which assessed respondents according to the interview data they provided and (2) establishing concurrence between observer assessments on the DI-ORS and the respondents' self-ratings on the DI scale. Interrater reliability of the DI-ORS was achieved with 97.9% agreement. However, there were significant differences (p < or = .05, 2-tailed) or disagreement on 5 out of 6 possible occurrences for agreement or disagreement between observer ratings on the DI-ORS and respondents' self ratings on the DI scale. The findings are informative for researchers who wish to use qualitative methods for construct validation.
Related Concept Videos
What are Estimates?
The estimate for the mean of a sample is denoted by ͞x, whereas the mean of the population is designated as μ. Further, parameters such as the mean,...
Measures of Intelligence
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this; it...
Qualitative Analysis
There are two main approaches to qualitative analysis:...
Data Validation
Nursing assessment guides are generally based on holistic models rather than medical...
Data Validation
Key parameters for method validation include:
Statistical Analysis: Overview
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...