A study of the effects of frequency of probe data collection and graph characteristics on teachers' visual analysis

Teachers often rely on visual analysis of graphed student performance data to evaluate progress and make program decisions. However, because collecting data can be time consuming and interfere with instruction, teachers would like to know how much data is necessary to make reliable judgments. To investigate the effect of frequency of data collection on teachers' judgments and decisions, this study addressed the question of whether teachers' judgments differ according to frequency of data collection, whether teachers' judgments differ according to type of trend, and whether teachers' judgments based on different types of graphs vary with frequency of data collection. A set of 16 graphs of actual student performance data was analyzed by 59 teachers of students with moderate to profound handicaps. The data were analyzed by a two-factor repeated measures design, and results indicated that when asked to evaluate student performance, teachers' judgments tended to be consistent and accurate when the graphed data represented continuous and systematic improvement in performance. However, when the data represented a decrease in performance, no change, or highly variable performance, judgments tended to differ by frequency. When asked to make program recommendations, teachers' judgments tended to differ by frequency for all types of trends.