Assessing image quality in photoacoustic imaging: A metric-based and deep learning-based evaluation

Photoacoustic (PA) imaging offers high-resolution functional imaging in vivo, but image quality varies with acquisition hardware, reconstruction methods, and scanning conditions. Reliable image quality assessment (IQA) is therefore critical for evaluating new PA technologies and ensuring reproducible research. IQA metrics originally proposed for natural images are widely used; however, their relevance to PA imaging has not been systematically benchmarked. We present the first large-scale benchmark of IQA in PA imaging, evaluating 11 full-reference (FR) metrics and 2 no-reference (NR) metrics across nearly one million PA images from five independent datasets. These datasets span phantoms, preclinical small-animal scans, and challenging ex vivo and in vivo acquisitions with controlled degradations across multiple commercial imaging systems. Metric scores were analyzed statistically to determine which metrics best distinguish differences in image quality. Metrics that showed consistent performance across datasets are modeled using three deep learning architectures (PAQNet, IQDCNN, and EfficientNetIQA), where we trained the models to predict these metric values directly from PA images, enabling automated no-reference quality estimation. The results show that structural similarity-based FR metrics, especially Structural Similarity Index Measure (SSIM) and its variants, reliably capture image quality differences in PAI, whereas conventional metrics like Peak Signal-to-Noise Ratio (PSNR) and existing NR metrics correlate poorly with reconstruction improvements. Deep learning models, particularly PAQNet, achieved high correlations with reference-based scores and provide a practical path to reference-free quality assessment, although generalization across datasets from different systems remains challenging. This study establishes a benchmark for PA image quality evaluation and demonstrates that structure-aware metrics and learned no-reference predictors can enable more reliable and automated quality assessment in PAI. We provide code for IQA metrics evaluation and deep learning models for reproducibility and further development at https://github.com/MellevdB/photoacoustic-image-quality-assessment.git.