A Novel Oppositional Chaotic Flower Pollination Optimization Algorithm for Automatic Tuning of Hadoop Configuration...

Area of Science:

• Computer Science
• Data Science
• Algorithm Optimization

Background:

• Big data era generates vast amounts of data, necessitating efficient big data platforms like Hadoop.
• Tuning numerous parameters on these platforms is complex, time-consuming, and challenging for performance optimization.
• Existing auto-tuning methods often increase computation time and reduce cluster efficiency.

Purpose of the Study:

• To propose a novel automatic parameter tuning system, Opt. Tuner, for Hadoop.
• To reduce computational and processing time for parameter tuning in big data systems.
• To enhance system performance through efficient parameter optimization.

Main Methods:

• Developed Opt. Tuner, an automatic system for selecting Hadoop configuration parameters.
• Utilized the Flower Pollination Algorithm for optimization.
• Introduced Opposition-Based Learning and chaotic mapping for population initialization, creating an Oppositional Chaotic Flower Pollination Algorithm.

Main Results:

• The proposed Oppositional Chaotic Flower Pollination Algorithm initializes populations effectively, guiding the search agent faster.
• Opt. Tuner successfully tunes 15 configuration parameters for Hadoop.
• Evaluated performance using wordcount and sort applications to demonstrate efficiency.

Conclusions:

• Opt. Tuner offers an efficient solution for automatic Hadoop parameter tuning.
• The novel Oppositional Chaotic Flower Pollination Algorithm improves search speed and individual generation.
• This approach enhances the performance and proficiency of big data systems like Hadoop.