Enhanced Sign Language Recognition Using Weighted Intrinsic-Mode Entropy and Signer's Level of Deafness

Sign language (SL) forms an important communication canal for the deaf. In this paper, enhanced SL recognition, by relating the individual way of signing with the signer's level of deafness (LoD) through a novel hybrid adaptive weighting (HAW) process applied to surface electromyogram and 3-D accelerometer data, is proposed. Using a LoD-driven genetic algorithm, HAW optimally weights the intrinsic modes of the acquired signals, preparing them for sample entropy (SampEn) estimation that follows. The resulting feature set, namely, weighted intrinsic-mode entropy (IMEn) (wIMEn), aims at increasing the SL-sign-classification accuracy alone or boosted by signer identification and/or signer's LoD-based group identification. The wIMEn was compared with three other feature sets, i.e., time frequency, SampEn, and IMEn, regarding their discrimination ability (both among signers and SL signs). Data from the dominant hand of nine subjects with various LoD were analyzed for the classification of 61 Greek SL (GSL) signs. Experimental results have shown that the introduced wIMEn feature set exhibited higher performance compared to others, both in signer identification and signer's LoD-based group identification and in GSL sign classification. The findings suggest that LoD could be considered in the construction of a signer-independent SL-classification system toward the enhancement of its performance.