Large language model prompt engineering for medical education: A practical guide for the Australian and New Zealand...

Large language models (LLMs) can achieve passing scores in specialist-level examinations, yet their capacity to author high-stakes examination content remains under-explored. Compared with published human benchmarks, LLMs create questions roughly 10 times faster, and contextual memory across sessions enables rapid diversification of topic coverage. Prompt design therefore emerges as an academic, not merely technical, craft. This article synthesises the emerging literature on artificial intelligence-assisted item writing and illustrates, through stepwise experimentation with OpenAI GPT-4o, how deliberate prompt engineering can align LLM output with the high standards of medical examinations. Key strategies explored include defining clinical context, imposing structural constraints, supplying exemplar items, assigning examiner roles, sequencing chain-of-thought instructions and requesting rationales. In a worked example, these approaches are layered sequentially while generating short-answer questions mapped to Australian and New Zealand College of Anaesthetists curriculum statements. An evidence-based approach to LLM use for question generation could markedly reduce examiner workload, provide educational integrity and enrich item banks.