Modeling Response Time and Responses in Multidimensional Health Measurement

Area of Science:

• Psychometrics
• Health Measurement
• Computerized Adaptive Testing

Background:

• The Activity Measure for Post-Acute Care (AM-PAC) uses Likert-type items across three scales.
• Accurate scoring requires appropriate item response theory (IRT) models, such as the multidimensional graded response model (MGRM).
• Large item banks necessitate robust calibration strategies for effective computerized adaptive testing (CAT).

Purpose of the Study:

• To explore calibrating a large item bank for multidimensional health measurement using IRT.
• To investigate the utility of incorporating item response time (RT) data alongside traditional response data.
• To evaluate the impact of interviewer effects on measurement precision.

Main Methods:

• Utilized the multidimensional graded response model (MGRM) for Likert-type item data.
• Adopted van der Linden's hierarchical modeling framework for concurrent calibration across data batches.
• Modeled response time (RT) using a lognormal distribution within a hierarchical framework.
• Compared models with and without interviewer covariates and item-interviewer interactions.

Main Results:

• A hierarchical model incorporating proportional item-interviewer interaction and a lognormal RT model provided the best fit.
• Including response time (RT) data did not significantly alter item parameter estimates or their standard errors.
• Response time (RT) data significantly reduced the standard error of latent trait estimates for patients.
• Including interviewer as a covariate did not yield further improvements in measurement precision.

Conclusions:

• Concurrent calibration using response time (RT) data enhances the precision of patient trait estimates in multidimensional health measures.
• The findings support the integration of RT data in computerized adaptive testing (CAT) for improved efficiency and accuracy.
• Interviewer effects, while present, were best modeled through interaction and did not necessitate separate covariate inclusion when RT was used.