On efficient meta-filtering of big data

There is an explosion in the number of new approaches being developed for extracting information from biological data, creating a need for intelligent ways to utilize so many methods. We take a perspective based on viewing methods as filters which reject undesired data and which may have complementary and redundant performance. We consider approaches for efficiently combining such filters. We provide quantitative strategies for choosing which filters to use and the best order to apply them, based on viewing each filter as a coordinate transformation on performance metrics. The approach is demonstrated using a range of methods to filter sequence data for homology detection, where we show the advantages of more sophisticated strategies in terms of achieving competitive speed with significantly-improved rejection of undesired data.