Keyword-augmented and semi-automatic generation of FESS reports: a proof-of-concept study

Surgical reports are usually written after a procedure and must often be reproduced from memory. Thus, this is an error-prone, and time-consuming task which increases the workload of physicians. In this proof-of-concept study, we developed and evaluated a software tool using Artificial Intelligence (AI) for semi-automatic intraoperative generation of surgical reports for functional endoscopic sinus surgery (FESS).

A vocabulary of keywords for developing a neural language model was created. With an encoder-decoder-architecture, artificially coherent sentence structures, as they would be expected in general operation reports, were generated. A first set of 48 conventional operation reports were used for model training. After training, the reports were generated again and compared to those before training. Established metrics were used to measure optimization of the model objectively. A cohort of 16 physicians corrected and evaluated three randomly selected, generated reports in four categories: "quality of the generated operation reports," "time-saving," "clinical benefits" and "comparison with the conventional reports." The corrections of the generated reports were counted and categorized.

Objective parameters showed improvement in performance after training the language model (p < 0.001). 27.78% estimated a timesaving of 1-15 and 61.11% of 16-30 min per day. 66.66% claimed to see a clinical benefit and 61.11% a relevant workload reduction. Similarity in content between generated and conventional reports was seen by 33.33%, similarity in form by 27.78%. 66.67% would use this tool in the future. An average of 23.25 ± 12.5 corrections was needed for a subjectively appropriate surgery report.

The results indicate existing limitations of applying deep learning to text generation of operation reports and show a high acceptance by the physicians. By taking over this time-consuming task, the tool could reduce workload, optimize clinical workflows and improve the quality of patient care. Further training of the language model is needed.