Modeling items for text comprehension assessment using confirmatory factor analysis.

Area of Science:

• Cognitive Psychology
• Educational Psychology
• Psycholinguistics

Background:

• Reading comprehension is a complex cognitive process crucial for understanding written text.
• Existing models often assume distinct processing levels but lack empirical validation in comprehension assessment.
• Current reading research frequently treats comprehension as a uni-dimensional construct, potentially oversimplifying the process.

Purpose of the Study:

• To investigate text comprehension as either a uni-dimensional or multi-dimensional construct using confirmatory factor analysis (CFA).
• To evaluate the efficacy of traditional one-shot assessment items in capturing the multifaceted nature of reading comprehension.
• To provide empirical evidence for distinct representational levels in text comprehension.

Main Methods:

• 400 participants read one of three short stories.
• Comprehension was assessed using 16 wh-questions (micro/inference) and 60 yes/no statements (micro/inference), alongside 16 main content extractions from summaries (macro-level).
• Confirmatory factor analysis (CFA) was employed to compare a one-dimensional model against a three-dimensional model (micro, inference, macro levels).

Main Results:

• A significant fraction of the compiled assessment items exhibited unsatisfactory psychometric properties.
• The three-factor model, representing distinct representational levels, consistently demonstrated a better fit than the uni-factor model.
• This provides strong evidence that text comprehension is a multi-dimensional construct.

Conclusions:

• Text comprehension is best conceptualized as a multi-dimensional construct, encompassing at least micro, inference, and macro-level representations.
• Traditional one-shot assessment items may not adequately capture the complexity of reading comprehension.
• The findings offer guidance for developing more robust and nuanced measures of reading comprehension in research.