Garbage In, Garbage Out: Context Engineering for Generating Multiple-Choice Questions for Medical Education Using...

Multiple-choice questions (MCQs) are a cornerstone of medical education, but generating high-quality items automatically remains challenging. We present KiMED, an LLM-based platform for automated MCQ generation in German, enhanced through Knowledge Graph (KG)-assisted retrieval. Biochemistry course materials - including book chapters, staff-written sections, and slide transcripts - were structured into subtopics, from which entities, properties, and relationships were extracted to construct a KG. The KG supports multi-agent MCQ generation by providing precise, contextually relevant information for stems, keys, and distractors - the question, correct and wrong answers. Evaluation of selected subtopics showed that 87% of entities and 82% of relationships were accurately represented, and 45% of KG-based MCQs were deemed usable by experts, compared to 23% from unstructured text. These results indicate that structured context significantly improves MCQ quality, though reliable question generation still requires optimization across context, agent workflows, and post-processing.