Apriori Versions Based on MapReduce for Mining Frequent Patterns on Big Data.

Area of Science:

• Data Mining
• Big Data Analytics
• Distributed Computing

Background:

• Pattern mining is crucial for extracting insights from raw data.
• Existing pattern mining techniques struggle with the scale of big data.
• The MapReduce framework offers a scalable solution for distributed data processing.

Purpose of the Study:

• To propose novel and efficient pattern mining algorithms tailored for big data environments.
• To leverage the MapReduce framework and Hadoop for enhanced pattern discovery.
• To address the performance limitations of traditional algorithms on massive datasets.

Main Methods:

• Development of three groups of MapReduce-based algorithms: Apriori MapReduce (AprioriMR), iterative AprioriMR, space pruning AprioriMR, top AprioriMR, and maximal AprioriMR.
• Implementation utilizing the Hadoop open-source framework.
• Performance evaluation on diverse big data datasets with up to 3×10^18 transactions and over 5 million distinct items.

Main Results:

• The proposed MapReduce-based algorithms show significant efficiency gains for complex big data pattern mining tasks.
• AprioriMR variants effectively handle large-scale datasets, outperforming established algorithms.
• The study highlights the unsuitability of the MapReduce paradigm for small data scenarios.

Conclusions:

• MapReduce-based pattern mining algorithms are a viable and efficient solution for big data challenges.
• The proposed algorithms offer improved performance and scalability for extracting meaningful patterns.
• Careful consideration of data size is necessary when selecting distributed computing paradigms for pattern mining.