Optimisation of a computer vision system for the interpretation of American Sign Language

Presented in this paper is a simulation algorithm for the optimisation of camera position with respect to the signer, to have a full and reliable interpretation of the American Sign Language. The simulation includes a three-dimensional world point into two-dimensional image point transformation algorithm, the effect of the depth information loss and a sign projection correction test. It is concluded that the viewing camera should be positioned at any point in a specified area subtended by a solid angle of 30 degrees, where the centre of the area is located at 45 degrees in the azimuth and 45 degrees in elevation relative to the signer. The theory and the technique are tested with regard to the efficiency of interpreting American Sign Language (ASL) by two adult signers. One of the signers had been using ASL on a regular basis since infancy, and the second signer had signed for the past five years. It is demonstrated that positioning the camera anywhere in the specified area provides a 96 per cent correct interpretation of the 36 signs tested. The results also provide a preliminary indication that signer variability may not present a major problem in interpretation, and that a computer vision system which captures the optimum depth information can distinguish between signs which, to the naked eye, appear to have similar characteristics.