Estimating Optimal Weights for Compound Scores: A Multidimensional IRT Approach.

Area of Science:

• Psychometrics
• Statistical Modeling
• Health Measurement

Background:

• Constructing reliable indices is crucial for accurate measurement in various fields.
• Latent variable modeling, particularly item response theory (IRT), provides a robust framework for reliability assessment.
• Existing methods may not optimally weight components for maximum reliability in complex scenarios.

Purpose of the Study:

• To propose a novel method for constructing indices with maximized reliability.
• To optimize component weights within a latent variable modeling framework.
• To compare different statistical approaches for estimating these optimal weights.

Main Methods:

• Developed a method to construct indices as linear functions of variables, maximizing reliability.
• Defined reliability using item response theory (IRT) principles.
• Proposed three weight estimation methods: marginal maximum likelihood (MML) and two Bayesian Markov chain Monte Carlo (MCMC) approaches (augmented and general Gibbs samplers).

Main Results:

• Simulation studies demonstrated the effectiveness of the proposed methods.
• The three estimation methods yielded very similar results.
• The general-purpose Gibbs sampler, being easily accessible, is recommended for practitioners.

Conclusions:

• The proposed method effectively constructs reliable indices by optimizing component weights.
• Bayesian MCMC methods provide reliable weight estimations comparable to likelihood-based approaches.
• The approach is applicable to complex real-world data, such as the 28-joint Disease Activity Score, across multiple time points and mixed data types.