Human Versus Nonhuman: Is There an App for That?

Artificial Intelligence (AI) has rapidly become a widespread technology, with many forms being freely accessible to anyone with internet access. Recently there has been an influx of digital applications, or apps, that claim to utilize the power of generative artificial intelligence (GAI) to identify everything from plant species to works of art based on a single photograph. Because these apps are emerging quickly and without oversight, their accuracy is largely unknown, and there are rarely disclaimers that indicate the probability of correctness for the app overall, or for a single result. We test a GAI-driven application that was originallymeant to assist medical students in their studies of human bone, but currently being used by some law enforcement to identify osseous remains as human or nonhuman. Thus, the goal of this project was to assess the accuracy of this application. Several test sets of known human and nonhuman skeletal elements were scanned into the application utilizing the digital devices integrated camera, as the application was intended for use. Three test sets of human and nonhuman skeletal remains were submitted to the application via image upload for analysis. The first test set consisted of 50 human and nonhuman bones typically submitted to the medical examiner's office for analysis. The second test set consisted of 5 nonhuman bones that were individually photographed and submitted in random order to the application five times, for a total of 25 images, to the application in a random order to test the application's repeatability. The final test set utilized 16 images of a nonhuman tibiotarsus from an Palaeognathae since it is similar in size to a human femur. Overall, this application performed poorly in a controlled laboratory setting on each of the three test sets. In the first test set, it correctly classified the name of the bone (eg, skull, humerus, "hip bone") seven out of 50 times (14%). It correctly identified nonhuman bones (by specifically stating that the bone was nonhuman) in three out of the 50 images (6%). It misclassified long bones (eg, identifying a humerus as a femur) 23 out of 50 images (46%). The second test of the app provided an overall accuracy of correct bone classification in 25 trials of 32%. In the third test set, fourteen of the sixteen images (87.5%) uploaded were misclassified as a presumably human "Femur," while the other two were unable to be classified. The authors advise caution when using applications like this one to assess material with evidentiary potential.