Performance Analysis of Bloom Filter for Big Data Analytics

Area of Science:

• Computer Science
• Data Science
• Information Retrieval

Background:

• The exponential growth of data from sources like social media and mobile apps defines "big data."
• Handling big data presents significant space-and-time efficiency challenges.
• Bloom filters (BF) are known for space-and-time efficiency but lack evaluation in big data contexts.

Purpose of the Study:

• To experimentally evaluate the effectiveness of Bloom filters (BF) for big data indexing and examination.
• To propose and evaluate a novel approach for reducing the false-positive rate associated with BF in big data scenarios.

Main Methods:

• Conducted an experimental study using large datasets to assess Bloom filter performance.
• Developed and tested a new method specifically designed to mitigate the false-positive rate of Bloom filters when applied to big data.

Main Results:

• Bloom filters demonstrated superior space-and-time efficiency for indexing and examining big data compared to existing methods.
• The novel false-positive rate reduction approach achieved a reduction of over 70%.

Conclusions:

• Bloom filters are a viable and efficient technique for managing big data.
• The proposed novel approach effectively addresses the challenge of high false-positive rates in big data applications of Bloom filters.