Evaluating the use of a novel expert system for interpretation of media authentication results

The evolution of artificial intelligence has enabled the creation of hyper-realistic synthetic images and videos, significantly complicating the authentication of media evidence. Many algorithmic approaches to media authentication have proven ineffective in digital forensics, particularly due to the difficulty of expressing results from black-box algorithms. Compounding this challenge, practitioners face long examination times and an ever-increasing volume of media evidence in modern investigations. To address these challenges, the authors developed an artificial intelligence expert system incorporating several logic-based authentication tests related to media file structure, attribute similarity analysis, and internal file attributes. These tests were organized into distinct authentication pathways to produce concise, natural language conclusions comparable to opinions rendered by human examiners. The automated nature of an expert system may allow for demonstrably accurate results at scale without extensive human resources. This preliminary study compared the accuracy of the artificial intelligence expert system's opinions with those of trained human examiners from diverse digital forensic backgrounds. Both the expert system and the examiners were provided the same dataset, consisting of original, transmitted, edited, and synthetic videos. Both groups received file structure data, attribute similarity analyses, proprietary structural data, and metadata values. Using a 14-question survey, responses were collected and evaluated for accuracy, and error rates were calculated. Across 20 respondents, the human mean score was 69% as compared to the expert system score of 91%. These results can be used to evaluate the appropriateness of deploying artificial intelligence expert systems for forensic examinations, alongside human interpretations.