Information utility: quantifying the total psychometric information provided by a measure.

Area of Science:

• Psychometrics
• Educational Measurement
• Psychological Assessment

Background:

• Traditional measures of test precision like reliability are sample-specific.
• The test information function offers detailed, sample-independent precision data but lacks an overall performance summary.
• Existing methods struggle to provide a comprehensive overview of a test's overall performance.

Purpose of the Study:

• Introduce the concept of information utility as a novel index for test performance.
• Provide a unified measure of psychometric information a test yields about a trait.
• Address limitations of existing metrics in summarizing overall test utility.

Main Methods:

• Defined information utility as an index quantifying psychometric information provided by a measure about a trait.
• Distinguished information utility from reliability and the test information function.
• Explored theoretical implications and applications of information utility.

Main Results:

• Information utility offers a sample-independent, overall summary of a test's psychometric information.
• This new metric integrates aspects of both reliability and test information function.
• Information utility is applicable across various psychometric contexts.

Conclusions:

• Information utility provides a valuable new metric for assessing overall test performance.
• It has broad implications for test selection, trait estimation, and computerized adaptive testing.
• This index is particularly useful when prior trait information is uncertain.