Related Experiment Video
Updated: Jul 9, 2025
Development of a Virtual Reality Assessment of Everyday Living Skills
Published on: April 23, 2014
Universal precautions required: Artificial intelligence takes on the Australian Medical Council's trial examination
Oliver Kleinig1, Joshua G Kovoor2, Aashray K Gupta3
1MBBS III, Faculty of Health and Medical Sciences, The University of Adelaide, Adelaide, SA; Royal Adelaide Hospital, Adelaide, SA.
Background And Objectives:
The potential of artificial intelligence in medical practice is increasingly being investigated. This study aimed to examine OpenAI's ChatGPT in answering medical multiple choice questions (MCQ) in an Australian context.
Method:
We provided MCQs from the Australian Medical Council's (AMC) medical licencing practice examination to ChatGPT. The chatbot's responses were graded using AMC's online portal. This experiment was repeated twice.
Results:
ChatGPT was moderately accurate in answering the questions, achieving a score of 29/50. It was able to generate answer explanations to most questions (45/50). The chatbot was moderately consistent, providing the same overall answer to 40 of the 50 questions between trial runs.
Discussion:
The moderate accuracy of ChatGPT demonstrates potential risks for both patients and physicians using this tool. Further research is required to create more accurate models and to critically appraise such models.
Related Concept Videos
01:29Issues And Trends In Healthcare Delivery System
Cost Containment
Payment for healthcare services has historically promoted adoption of costly and often unnecessary or inefficient...
01:26Standard Precaution
Hand hygiene is the most crucial means to prevent the transmission of disease. Employers are legally required to provide their workers with personal protective equipment (PPE) to minimize exposure or contact with...
01:30Current Trends in Nursing II