An intelligent point-of-care photoacoustic and ultrasound dual-modality imaging system for Rheumatoid arthritis in...

Aiming at an intelligent point-of-care imaging technology for rheumatology clinics, a fully automatic 3D photoacoustic (PA) and ultrasound (US) dual-modality system driven by a robot and powered by deep learning (DL)-based image processing was developed. Automated scanning of volumetric images from patient joints, plus DL-based tissue segmentation and quantification of imaging biomarkers, ensures that the measurements from this system are objective and reproducible. Clinical validation was conducted via a longitudinal study on 43 finger joints from patients affected by inflammatory arthritis. Using manual segmentations as the gold standard, our DL algorithm utilizing the 3D Deep Attentive Feature (DAF3D) model showed satisfactory performance in automatic segmentation of joint space and synovial region and achieved a Dice score of 0.77±0.03 and an IoU of 0.64±0.03. Based on the tissue segmentations facilitated by the DAF3D model, six volumetric imaging biomarkers reflecting the activity of arthritis and its change in response to the treatment were quantified, including hyperemia, blood oxygenation, US power Doppler, joint space echogenicity, joint space volume, and synovial volume. The imaging biomarkers quantified from DL-based segmentation and manual segmentation showed moderate to strong correlations ( $R$ : 0.41-0.86). Hyperemia quantified from PA imaging has the strongest association with the disease activity indicated by Clinical Assessment Questionnaire (CAQ) scores, with $R^2$ =0.41. Linear models combining the two biomarkers from PA imaging, the four biomarkers from US imaging, and all six imaging biomarkers rendered moderate to very strong associations with the disease activity scores, with $R^2$ of 0.41, 0.26, and 0.52, respectively.