Multi-laboratory performance assessment of diffuse optics instruments: the BitMap exercise.

Area of Science:

• Biophotonics
• Medical Imaging
• Optical Engineering

Background:

• Standardization and performance assessment are critical for clinical adoption of biophotonics.
• Multi-laboratory initiatives are essential for establishing universal instrument comparison protocols.
• Reference tissue phantoms are needed for reliable biophotonics instrument evaluation.

Purpose of the Study:

• To conduct the largest multi-laboratory comparison of near-infrared diffuse optics instruments.
• To assess instrument performance using consolidated protocols and tissue phantom kits.
• To identify key performance indicators and variations across different diffuse optics systems.

Main Methods:

• Involved 28 instruments from 12 institutions across eight experiments.
• Utilized three established protocols (BIP, MEDPHOT, NEUROPT) and three tissue phantom kits.
• Extracted 20 synthetic indicators, including novel definitions, to evaluate instrument performance.

Main Results:

• Observed substantial differences in hardware performance, including responsivity, dark count rate, and temporal resolution.
• Achieved agreement within 12% for homogeneous optical properties in half of the systems.
• Demonstrated temporal stability (<5% over 1h) and day-to-day reproducibility (<3%) for some systems.

Conclusions:

• This extensive exercise provides a detailed assessment for reference grading of near-infrared diffuse optical instruments.
• The findings highlight significant hardware variations impacting performance.
• The open-access dataset will enable further analysis of analysis tools and hardware impacts.