Display of merged multimodality brain images using interleaved pixels with independent color scales

This article reviews common methods for two-dimensional display of registered multimodality brain images and describes a software package for presentation of merged MRI and PET images that runs on a workstation with an eight-bit color display. The software package displays merged brain images from multiple modalities in a way that is readily manipulated, visually pleasing and easy to interpret. The display method used, i.e., interleaving of alternate pixels with independent color scales, is effective in producing merged images with high contrast-detail for each volume. Interleaving images from different volumes creates unusual perceptual effects, one of which is the apparent camouflage of low-contrast signals by high values in the paired volume.

The camouflage effect was thought to arise from perceptual merging of adjacent pixels. An observer experiment was conducted to investigate this tendency of high-activity PET data to obscure low-contrast detail in interleaved MRI data in spite of the digital independence of neighboring pixels. Four observers were presented with 20 combinations of signal plus background targets with uniform mask images, using a two-alternative forced-choice experimental design with 50 trials per combination.

The psychophysical evaluation of the ability of human observers to detect the simple test objects in an interleaved image presentation indicated a statistically significant camouflage effect of one volume on the other for some combinations of target and mask. The concept of perceptual merging of adjacent pixels was able to predict which combinations caused the greatest degradations in performance.

The image interleaving approach to the display of two-dimensional slices from registered image volumes makes efficient use of an eight-bit color display. Contrast resolution of both individual volumes is high compared with that in other techniques and the volumes are presented in familiar color scales. However, the method yields an unexpected camouflage effect that tends to obscure low-contrast signals. The practical effect of such camouflage on the interpretation of clinical images remains to be investigated.