Local self similar descriptors: comparison and application to gastroenterology images

Local descriptors coupled with robust methods for learning visual dictionaries have been a pivotal tool in computer vision. Although the identification of similar patterns is commonly conducted on some stage of the bag-of-words framework, a prior assessment of spatial local similarities can be indicative of specific objects, and thus improved recognition rates. In this work we delve a function of similarity for enhancing the discriminative power of local constrained SIFT descriptors. Motivated by gastrointestinal images where diagnosis through endoscopy plays a decisive role in cancer detection and resulting prognosis, visual cues in these early stages are slim and of difficult perception. In order to capture these patterns we propose a self-similarity approach (based on a neighbourhood analysis of SIFT descriptors) to assess local variances through a weight function. Based on extensive simulations our approach achieved a performance of 88%: 3% higher than the standard SIFT, 10% higher than Haar wavelet and 13% higher than LBPs.