Assessing ChatGPT's Reliability as an Educational Resource for Information on Breast Reduction Surgery

With advances in AI and machine learning, platforms like OpenAI's ChatGPT are emerging as educational resources. While these platforms offer easy access and user-friendliness due to their personalized conversational responses, concerns about the accuracy and reliability of their information persist. As one of the most common surgical procedures performed by plastic surgeons worldwide, breast reduction surgery (BRS) offers relief for the physical and emotional burdens of large breasts. However, like any surgical procedure, it can raise a multitude of questions and anxiety.

To evaluate the quality of medical information provided by ChatGPT in response to common patient inquiries about breast reduction surgery, we developed a 15-question questionnaire with typical patient questions about BRS. These questions were presented to ChatGPT, and the answers were compiled and presented to five board-certified plastic surgeons. Each specialist categorized the response as (1) Appropriate, the response accurately reflects current medical knowledge and best practices for BRS; (2) No, not thorough, the response lacks sufficient detail to be a helpful educational resource; (3) No, inaccurate, the response contains misleading or incorrect information.

A total of 75 survey responses were obtained, with five experts each analyzing 15 answers from ChatGPT. Of these, 69 (92%) responses were determined to be accurate. However, six (8%) responses were concerning to our experts: four (5.3%) lacked detail, and two (2.7%) were found to be inaccurate. Chi-square analysis revealed no statistical significance in the distribution of responses categorized as "accurate" versus "not thorough/inaccurate," and "not thorough" versus "inaccurate" (p=0.778 and p=0.306, respectively).

While ChatGPT can provide patients with basic background knowledge on BRS and empower patients to ask more informed questions during consultations, it should not replace the consultation and expert guidance of a board-certified plastic surgeon.