Detecting keypoints with semantic labels on skull point cloud for plastic surgery

Using deep learning models to automatically generate reference keypoints with surgical semantic labels and segment bone blocks in plastic surgery provides valuable preoperative planning references. This study aimed to develop a robust and precise keypoint detection framework for dense three-dimensional (3D) skull point clouds to assist in plastic surgery.

A keypoint descriptor-detector framework was proposed to address keypoint detection in dense point cloud models for facial plastic surgery. The keypoint descriptor identified potential keypoint areas on the point cloud model using the PointRes2Net module to initialize keypoints, which were further optimized by a keypoint detector constructing a self-organized map. Based on the detected keypoints, a new localized small-part segmentation strategy for dense point cloud models was introduced. A bounding box was generated by the detected keypoints, enclosing small bone blocks as the region of interest (ROI) for segmentation.

The mean squared error (MSE) between the keypoints detected on the point cloud using this framework and the ground truth has been reduced to 3.35 mm on the skull models with an average size of 231 mm × 173 mm × 151 mm, outperforming existing point cloud keypoint detection algorithms without requiring additional keypoint annotation on two-dimensional (2D) images for auxiliary training. Furthermore, the proposed framework's segmentation strategy demonstrated a 22.69% improvement in average precision compared to direct segmentation, with a 34.15% improvement in precision for smaller parts.

The proposed method accurately detects keypoints with surgical semantic labels on dense medical point clouds. Both keypoint detection and segmentation results align closely with the ground truth, providing valuable references for preoperative planning in plastic surgery.