Measuring academic production

The entire academic medical community is under increasing pressure to define and measure its activities. Previous relative value-based systems to measure research, teaching, administration, and patient care share several features that threaten their acceptability and validity. Using a bottom-up approach, our academic family medicine department attempted to develop a measurement system that avoided some of the flaws of the earlier systems.

The system was developed in two phases. In the first phase, faculty members were invited to submit lists of all their professional activities. In the second phase, the faculty rated the relative value of a comprehensive list of academic activities using an unbounded ratio scale and indicated how many times a year they did each activity.

Phase One resulted in a list of 96 academic activities. The activity rated in Phase Two as having the greatest relative value was principal investigator of a funded grant (relative value=30.23), followed by sole author of a book (relative value=28.25). The activity with the smallest relative value was attending a faculty meeting (relative value=.36). A half-day clinic session had a relative value of 1.08. The department's annual production, measured in relative value units, was 5,764 units of administration, 5,702 units of clinical activities, 5,480 units of teaching, and 4,401 units of scholarly activities.

Overall, the process efficiently produced relative value measures for a large number of faculty activities using a process in which most of the faculty participated. Problems with internal coherence, face validity, and inconsistencies in estimation suggest it would be premature to use such estimates of relative value to quantify individuals 'productivity as a basis for budgetary decisions.