Use of a Generative Pretrained Transformer to Answer Questions and Facilitate a Large Randomized Controlled Trial

Generative artificial intelligence may help facilitate clinical trials. We sought to determine whether a customized generative pretrained transformer (GPT) could assist clinical trial sites within a randomized, controlled trial with rapid responses to protocol and procedure-related questions.

Within a large clinical trial, Sleep SMART (Sleep for Stroke Management and Recovery Trial), we developed, tested, and implemented a customized GPT designed to answer, in real-time, procedure-related questions. This support was offered to all active trial sites and questions and responses were monitored by the central study team. An anonymous survey also queried primary study coordinators about their experiences with the GPT.

Of the 785 questions entered during a 10-month implementation period, 75% were able to be answered per the GPT. On manual review, of the 588 that the GPT reportedly answered, only 13 (2%) were not helpful responses, 12 (2%) were incomplete responses, 5 (1%) were misleading, and 3 (1%) contained the correct response but also provided some conflicting messaging. Of 95 primary study coordinators to whom the survey was sent, 45 (47%) responded. Of those, 21 (47%) reported having used the GPT. Of the 19 who provided more detailed information, 89% found it to be very helpful (n=11) or helpful (n=6). Most (79%) found the responses to be accurate (n=15) or partially accurate (n=2), and 89% were very satisfied (n=10) or satisfied (n=7) with the GPT.

This novel use of a customized GPT suggests it could be valuable in support of clinical sites within a large trial. Further research should confirm equivalent accuracy and safety through a direct comparison to human support.

URL: https://clinicaltrials.gov/; Unique Identifier: NCT03812653.