Assessing evidentiary value in fire debris analysis by chemometric and likelihood ratio approaches

Area of Science:

• Forensic Science
• Computational Chemistry
• Machine Learning

Background:

• Accurate classification of fire debris is crucial for arson investigations.
• Machine learning models offer potential for automated analysis of fire debris samples.
• Previous studies have explored various algorithms for ignitable liquid residue detection.

Purpose of the Study:

• To evaluate the performance of Support Vector Machine (SVM), Linear Discriminant Analysis (LDA), Quadratic Discriminant Analysis (QDA), and k-Nearest Neighbors (kNN) algorithms for binary classification of fire debris.
• To compare the effectiveness of these methods using both computationally generated (in silico) data and real fire debris samples.
• To assess the calibration and error rates of each method under different validation scenarios.

Main Methods:

• Binary classification using SVM, LDA, QDA, and kNN algorithms.
• Training data generated by computationally mixing ignitable liquid and substrate pyrolysis data.
• Validation performed on unseen in silico data and actual fire debris from large-scale burns.
• Calculation of class membership probabilities and likelihood ratios.

Main Results:

• SVM showed high discrimination and low error on in silico data but significantly decreased performance on real fire debris.
• QDA and kNN exhibited similar performance trends to SVM, with reduced effectiveness on actual fire debris.
• LDA demonstrated poorer discrimination on in silico data but maintained consistent performance on fire debris, with no significant deterioration.
• LDA presented higher error rates and slightly poorer calibration on in silico data compared to SVM.

Conclusions:

• While SVM performs well on simulated data, its performance degrades on real-world fire debris, suggesting limitations for forensic applications.
• LDA, despite initial lower performance on simulated data, proves to be a more robust and reliable method for classifying ignitable liquid residue in actual fire debris.
• The findings highlight the importance of validating computational models with real-world data in forensic science.
• LDA is recommended for practical application in fire debris analysis due to its stability across different data types.