An information-oriented paradigm in evaluating accuracy and agreement in radiology

Area of Science:

• Radiology
• Information Theory
• Medical Diagnostics

Background:

• Traditional diagnostic accuracy and inter-reader agreement metrics in radiology often lack robust information content.
• Commonly used metrics like sensitivity, specificity, and Cohen's kappa rely on confusion matrices, which may not fully capture diagnostic information.
• There is a need for advanced methodologies to quantitatively assess diagnostic performance and consistency.

Purpose of the Study:

• To present a methodological paradigm using Shannon's information theory to measure diagnostic accuracy and inter-reader agreement in radiology.
• To propose novel metrics derived from mutual information as an alternative to traditional assessment methods.
• To demonstrate the application of information theory in modeling the diagnostic process as an information flow.

Main Methods:

• Modeling the diagnostic process as a "diagnostic channel" and inter-reader agreement as an "agreement channel" based on information theory.
• Utilizing Shannon's mutual information to derive quantitative measures for accuracy and agreement.
• Comparing the proposed information-theoretic metrics with conventional metrics.

Main Results:

• Information theory-based metrics for diagnostic accuracy are independent of disease prevalence.
• Information theory-based metrics for inter-reader agreement can overcome limitations of Cohen's kappa.
• The proposed approach provides a novel framework for assessing diagnostic performance and consistency in radiology.

Conclusions:

• Information theory offers a powerful framework for modeling and quantifying diagnostic processes in radiology.
• The proposed information-theoretic metrics provide a valuable alternative for evaluating diagnostic accuracy and inter-reader agreement.
• This approach enhances the understanding and measurement of information flow in radiological decision-making.