A Diagnostic Facet Status Model (DFSM) for Extracting Instructionally Useful Information from Diagnostic Assessment

Area of Science:

• Educational Measurement
• Psychometrics
• Cognitive Science

Background:

• Modern educational reforms require improved diagnostic testing.
• Current methods often fail to capture intermediate understandings crucial for learning.
• Accurate interpretation of assessment results is vital for targeted instruction.

Purpose of the Study:

• To develop a novel psychometric model, the diagnostic facet status model (DFSM), for enhanced diagnostic assessment.
• To simultaneously model target and intermediate student understandings.
• To utilize all response options, including incorrect ones, for more nuanced diagnostic information.

Main Methods:

• Developed the diagnostic facet status model (DFSM), a type of cognitive diagnostic model (CDM).
• Employed a regularized expectation-maximization algorithm (REM) for model estimation, incorporating a log-penalty for sparsity.
• Proposed a heuristic algorithm to infer facet maps from estimated permissible latent classes.

Main Results:

• The regularized expectation-maximization algorithm (REM) effectively estimated model parameters and identified permissible latent classes in simulations.
• The DFSM successfully modeled both target and intermediate facets of understanding.
• Analysis of response options, not just right/wrong, provided unique diagnostic insights.

Conclusions:

• The diagnostic facet status model (DFSM) offers a more nuanced approach to educational assessment.
• The proposed REM algorithm is effective for estimating the DFSM parameters.
• The model demonstrates feasibility for real-world data analysis, improving diagnostic accuracy.