Feature engineering solution with structured query language analytic functions in detecting electricity frauds using...

Area of Science:

• Data Science
• Machine Learning
• Energy Systems

Background:

• Electricity fraud detection faces challenges due to unreliable datasets with missing records and meter reading errors.
• Real-world datasets often exhibit data inconsistencies, faults, and misinterpretations, complicating accurate fraud identification.

Purpose of the Study:

• To develop and evaluate an effective electricity fraud detection system using real Tunisian electricity consumption data.
• To address the challenges of unreliable data, weakly correlated features, and highly unbalanced datasets in fraud detection.

Main Methods:

• Employed extensive feature engineering utilizing structured query language (SQL) analytic functions.
• Implemented a hybrid approach combining classifiers with an unsupervised anomaly detection algorithm (Isolation Forest).
• Utilized double dataset merging to uncover additional data dimensions for improved irregularity detection.

Main Results:

• Feature processing techniques significantly enhanced the Area Under the Curve (AUC) score for the Decision Tree algorithm from 0.68 to 0.99.
• The proposed methods demonstrated effectiveness in detecting irregularities within large, complex datasets.
• Machine learning algorithms successfully managed weakly correlated features and imbalanced datasets.

Conclusions:

• The combination of extensive feature engineering with SQL analytic functions and anomaly detection provides a robust solution for electricity fraud detection.
• The developed approach offers improved accuracy and reliability in identifying fraudulent electricity consumption, even with challenging data characteristics.
• This research provides a valuable framework for utility companies to combat electricity theft more effectively.