Can artificial intelligence accurately detect and summarize anatomy education literature? A comparative analysis of...

Artificial intelligence platforms have been suggested as tools that can facilitate anatomy teachers' work and students' learning process. We aimed to investigate the ability of ChatGPT to detect and summarize studies of the anatomy education literature compared to ScholarGPT, a version of ChatGPT specified in academic research. Secondly, we aimed to explore if the ability of each platform is influenced by the level of queries complexity.

We asked the two platforms to list five studies about each of the following three topics: (1) use of virtual reality in anatomy education, (2) use of stereoscopic virtual reality in anatomy education, (3) use of stereoscopic virtual reality in anatomy education, involving user's interaction with the virtual environment. We assessed if the retrieved studies fulfilled the search criteria, and if their summaries were accurate (if they contained true information about all the educational results of the article's abstract).

The ChatGPT's percentages of successful detection were 100%, 60% and 0% respectively for the three queries. The percentages of accurate summaries were 60%, 20% and 0% respectively. ScholarGPT performed better, with a percentage of successful detection 100%, 60% and 40% respectively. The percentages of accurate summaries were 80%, 60% and 40% respectively. Both platforms showed bias in favor of the educational intervention.

ChatGPT and ScholarGPT are not currently at an adequate level to essentially aid researchers to detect and summarize studies of the anatomy education literature. Ongoing research may increase the ability of those platforms to provide more reliable information.