Measuring the quality of quality measures

Area of Science:

• Image Processing
• Human-Computer Interaction
• Metrology

Background:

• Print quality (PQ) is a perceptual attribute, typically assessed via human surveys.
• Human surveys for PQ are time-consuming, necessitating automated evaluation tools.
• Qualifying automated PQ tools requires methods that account for human opinion variability.

Purpose of the Study:

• To develop a robust method for qualifying automated print quality evaluation tools.
• To model human quality preference to derive an appropriate qualification methodology.
• To address the challenge of conflicting human opinions in tool evaluation.

Main Methods:

• Modeling human quality preference.
• Deriving a qualification method that incorporates inter-observer variability.
• Applying the derived method to qualify a mechanical band meter.

Main Results:

• A novel method for qualifying automated quality evaluation tools was derived.
• The method effectively handles disagreements among human assessments.
• Demonstrated the qualification of a mechanical band meter using the proposed method.

Conclusions:

• A statistically sound method is crucial for validating automated PQ tools.
• The derived method provides a reliable approach to qualify PQ evaluation tools.
• This work offers a standardized approach for assessing automated PQ assessment tools.