Assessing technician effects when extracting quantities from microscope images

Consider an experiment where the response is based on an image; e.g., an image captured to a computer file by a digital camera mounted on a microscope. Suppose relevant quantitative measures are extracted from the images so that results can be analyzed by conventional statistical methods. The steps involved in extracting the measures may require that the technicians, who are processing the images, perform some subjective manipulations. In this case, it is important to determine the bias and variability, if any, attributable to the technicians' decisions. This paper describes the experimental design and statistical analyses that are useful for those determinations. The design and analysis are illustrated by application to two biofilm research projects that involved quantitative image analysis. In one investigation, the technician was required to choose a threshold level, then the image analysis program automatically extracted relevant measures from the resulting black and white image. In the other investigation, the technician was required to choose fiducial points in each of two images collected on different microscopes; then the image analysis program registered the images by stretching, rotating, and overlaying them, so that their quantitative features could be correlated. These investigations elucidated the effects of the technicians' decisions, thereby helping us to assess properly the statistical uncertainties in the conclusions for the primary experiments.