Testing and Validating Machine Learning Classifiers by Metamorphic Testing.

Area of Science:

• Software Engineering
• Artificial Intelligence
• Machine Learning

Background:

• Machine learning algorithms are crucial in domains like bioinformatics and computational linguistics.
• Detecting faults in machine learning applications is challenging due to the absence of a 'test oracle' for output verification.

Purpose of the Study:

• To present a novel technique for testing machine learning classification algorithm implementations.
• To address the software quality and fault detection challenges in machine learning applications.

Main Methods:

• The study employs metamorphic testing, a technique proven effective in overcoming the oracle problem.
• Mutation analysis and cross-validation were conducted to evaluate the method's effectiveness.

Main Results:

• Metamorphic testing demonstrated high effectiveness in identifying mutants.
• Cross-validation alone proved insufficient for detecting faults in supervised classification programs.
• The technique successfully identified real faults in an open-source classification program.

Conclusions:

• Metamorphic testing is a viable and effective approach for testing machine learning classification algorithms.
• Programmers can benefit from insights into common pitfalls when implementing machine learning algorithms.