Camera-based detection and removal of shadows from interactive multiprojector displays

Front-projection displays are a cost-effective and increasingly popular method for large format visualization and immersive rendering of virtual models. New approaches to projector tiling, automatic calibration, and color balancing have made multiprojector display systems feasible without undue infrastructure changes and maintenance. As a result, front-projection displays are being used to generate seamless, visually immersive worlds for virtual reality and visualization applications with reasonable cost and maintenance overhead. However, these systems suffer from a fundamental problem: Users and other objects in the environment can easily and inadvertently block projectors, creating shadows on the displayed image. Shadows occlude potentially important information and detract from the sense of presence an immersive display may have conveyed. We introduce a technique that detects and corrects shadows in a multiprojector display while it is in use. Cameras observe the display and compare observations with an expected image to detect shadowed regions. These regions are transformed to the appropriate projector frames, where corresponding pixel values are increased and/or attenuated. In display regions where more than one projector contributes to the image, shadow regions are eliminated.