Automatic discovery and classification of bioinformatics Web sources

The World Wide Web provides an incredible resource to genomics researchers in the form of query access to distributed data sources--e.g. BLAST sequence homology search interfaces. The number of these autonomous sources and their rate of change outpaces the speed at which they can be manually classified, meaning that the available data is not being utilized to its full potential. Manually maintaining a wrapper library will not scale to accommodate the growth of genomics data sources on the Web, challenging us to produce an automated system that can find, classify and wrap new sources without constant human intervention. Previous research has not addressed the problem of automatically locating, classifying and integrating classes of bioinformatics data sources.

This paper presents an overview of a system for finding classes of bioinformatics data sources and integrating them behind a unified interface. We describe our approach for automatic classification of new Web sources into relevance categories that eliminates the human effort required to maintain a current repository of sources. Our approach is based on a meta-data description of classes of interesting sources that describes the important features of an entire class of services without tying that description to any particular Web source. We examine the features of this format in the context of BLAST sources to show how it relates to Web sources that are being described. We then show how a description can be used to determine if an arbitrary Web source is an instance of the described service. To validate the effectiveness of this approach, we have constructed a prototype that correctly classifies approximately two-thirds of the BLAST sources we tested. We conclude with a discussion of these results, the factors that affect correct automatic classification and areas for future study.