Performance evaluation tests of new laboratory procedures or devices: practical reporting criteria

Area of Science:

• Laboratory Medicine
• Medical Device Evaluation
• Scientific Publishing

Background:

• Increasing number of published evaluations for new laboratory procedures and devices.
• Shift towards commercial systems and away from laboratory-based systems.
• Potential users rely on peer opinions for system functionality assessments.

Purpose of the Study:

• Provide guidance for creating valuable evaluation reports of laboratory systems.
• Address the lack of statistical rigor in many published evaluations.
• Improve the practical utility of peer-reviewed assessments for new laboratory technologies.

Main Methods:

• Review of common statistical deficiencies in current evaluation reports.
• Identification of best practices for statistical analysis in performance evaluations.
• Recommendations for sample size determination in laboratory studies.

Main Results:

• Many published reports lack adequate statistical guidance.
• Frequent use of small sample numbers leading to unreliable significance probabilities.
• Recitative reporting of percentage differences without sufficient statistical context.

Conclusions:

• There is a critical need for improved statistical methodology in laboratory system evaluations.
• Adherence to statistical guidance will enhance the practical value of published reports.
• Future evaluations should prioritize robust statistical design and reporting for better peer assessment.