Agreement and reproducibility of subjective methods of measuring faculty time distribution

The authors performed this study to compare self-reported estimates of the time radiology members spend performing various activities between reporting methods and academic years.

For 3 consecutive academic years, the percentage of time each faculty member reported spending on clinical, teaching, research, and service and/or administrative activities was noted on each of three separate reports: a quarterly report to department administration, an annual report to university administration, and the annual review with his or her department supervisor. For each activity, the means were compared between methods and between years.

In general, the year-to-year changes in mean percentage for each activity and method were not statistically significant (34 of 36 comparisons). Nineteen of 36 comparisons, however, showed significant differences between reporting methods for a particular activity. For example, the mean percentages obtained with the three methods from 1999 to 2000 varied from 49% to 66% (clinical), 14% to 34% (teaching), 8% to 15% (research), and 3% to 11% (service and/or administrative).

Current methods used to quantify faculty time distribution yield significantly different results despite internal consistency from year to year for each method. New and clearer methods for determining faculty time distribution are needed.