Assessing portal design skills in the radiation oncology interactive case management examination

The American Board of Radiology is developing a computerized interactive case management examination to be used to evaluate the clinical skills of radiation oncologists. In the past, these skills have been evaluated by a pencil and paper written examination and an oral examination. With the increasing capabilities of computers, these skills can be easily, and perhaps even better, evaluated digitally. The aim is to develop an examination, which will be based on actual clinical cases, and be interactive so that it better mimics the clinical practice of a radiation oncologist than a written examination. It will also be less labor-intensive and less expensive than an oral examination. One of the most important skills of a radiation oncologist is the ability to design treatment portals that will encompass the entire cancer and yet minimize the irradiation of critical tissues and normal organs. Important parameters for radiation oncologists include the direction of the treatment beam, the size and shape of the portals, and the location of the margins of the field relative to patient anatomy and tumor location. In order to evaluate a physician's ability to design treatment portals, the computer-based examination has the capability to interactively construct field lines. The computer interface allows the candidate to draw field lines on a digitized x-ray image in a manner similar to practice. After the candidate illustrates the field lines, the evaluation of the response must be performed quickly to avoid interrupting the flow of the examination. The answer key is stored as a lossless compressed image. The key contains three regions consisting of (1) the must include region, which contains the tumor; (2) the must-exclude region, which contains tissues that if damaged would affect patient vitality and quality of life; and (3) the envelope of acceptable curves. Each region is assigned a unique byte code. The candidate's response is assigned a fourth byte code. Using basic logic operations, the response is swiftly evaluated. The scoring algorithm scores a candidate's action as correct if his/her drawn area encompasses all of the "must-include region" and is within the "envelope of acceptable curves." It scores a candidate's action as incorrect if his/her drawn area overlaps any part of the "exclude region" and/or exceeds at any point the "envelope of acceptable curves."