Video dataset for surgical phase, keypoint, and instrument recognition in laparoscopic surgery (PhaKIR)

This data article describes the Surgical Procedure Phase, Keypoint, and Instrument Recognition (PhaKIR) dataset, a collection of eight complete laparoscopic cholecystectomy videos acquired during routine minimally invasive procedures at three German medical centers. The recordings were captured with different monocular endoscopic camera systems at 25 frames per second and a resolution of 1920 × 1080 pixels, with durations ranging from 28 to 58 min. Segments showing regions outside the abdominal cavity were removed for anonymization, and the corresponding cut indices are provided. The dataset offers unified annotations for three interrelated tasks on the same complete surgical sequences: surgical phase recognition, annotated for every frame at 25 fps (485,875 frames), instrument keypoint estimation (19,435 frames), and instrument instance segmentation (19,435 frames), the latter two annotated at one frame per second (every 25th frame). Surgical phases follow the seven-phase Cholec80 scheme, extended by an undefined label for transitional frames. The instrument annotations distinguish 19 instrument classes as well as individual instances of the same class, with keypoint annotations comprising two to four class-dependent points labeled with COCO-style visibility states. Segmentation and keypoint labels were created manually using the Computer Vision Annotation Tool (CVAT), whereas phase labels were derived from documented phase-transition timestamps, and all annotations underwent a multi-stage review by a medically trained team. The data are provided in open formats (MP4, CSV, JSON, PNG) together with a frame-extraction script under the CC BY-NC-SA license and are available through controlled access upon request via the Zenodo platform. Because the dataset combines procedural context, instrument pose, and pixel-accurate instance segmentations within full-length recordings from multiple institutions, it can be reused for single-task or multi-task model development, for temporally aware approaches that exploit motion continuity, and for cross-institutional generalization protocols such as leave-one-hospital-out evaluation. The dataset served as the training resource for the PhaKIR Challenge at the Endoscopic Vision (EndoVis) Challenge at MICCAI 2024.