Enhanced bi-branch deep learning network for in vivo hyperspectral imaging recognition of organs and tissues

Hyperspectral imaging, as an emerging medical imaging technology, offers significant potential in biomedical research due to its ability in capturing rich spectral information. An enhanced bi-branch network integrating graph convolutional network (GCN) and transformer for hyperspectral image recognition of multiple organs and tissues was developed. GCN is able to extract regional information of same category adjacent to the target pixel, while transformer utilizes long-range dependencies to capture the boundary information. Furthermore, feature enhancement modules were incorporated to improve the performance of the model, and the efficiency was shown by a comparison of the feature distributions obtained with and without these modules. Finally, the model was validated using a public porcine dataset and was applied to in vivo organ and tissue identification from the hyperspectral images of zebrafish measured by diffuse reflectance near-infrared spectroscopy. The proposed model was found able to achieve a higher precision in recognizing the organ and tissue structures with clear and continuous boundaries.