Prompt engineering experiment on ChatGPT's ability to recommend orthopedic surgeons

People often use search engine results and online reviews to find health services and medical practitioners. However, it can be overwhelming to choose a practitioner when given hundreds of physician listings, making it tempting for people to use large language models (LLMs) such as ChatGPT to provide doctor recommendations. The present study examines ChatGPT's ability to provide recommendations for real medical practitioners (i.e., orthopedic surgeons) and experimentally tests how the inclusion of patient characteristics (e.g., age, race, income) in prompt queries impacts responses. Out of 40,500 queries, ChatGPT stated that it was unable to provide recommendations for 52.8 % of responses. Results show that ChatGPT varied its recommendation response depending on personal characteristic of tested patient personas (e.g., race). Patient characteristics most relevant to healthcare access - income, health insurance status, and location - were the strongest predictors on whether ChatGPT provided a surgeon recommendation. Specifically, prompts where the patient persona had a high income and stated they had health insurance were significantly more likely to receive recommendations. Further, patient prompts based in New York City and Chicago were more likely to receive a recommendation compared to Phoenix and Houston. A subset of coded responses (n = 1000) show that only 44.6 % of recommended surgeons were valid. Among the valid surgeons, 97.1 % (n = 433) were male and 81.4 % (n = 363) were White. Our findings show that ChatGPT does not reliably provide valid surgeon recommendations and suggests it may be biased when given personal descriptions of patients when making queries.