Artificial neural networks can distinguish novice and expert strategies during complex problem solving

To determine whether expert problem-solving strategies can be identified within a large number of student performances of complex medical diagnostic simulations.

Self-organizing artificial neural networks were trained to categorize the performances of infectious disease subspecialists on six computer-based clinical diagnostic simulation that used the sequence of diagnostic tests requested as the input data. Six hundred seventy-six student solutions to these problems were presented to these trained neural networks to determine which, if any, of the student solutions represented those of the experts.

For each simulation, the expert performances clustered around one dominant output neurode, indicating that there were common problem-specific features associated with the experts' problem-solving performances. When the performances of students who also made correct problem diagnoses were tested on these expert-trained neural networks, 17% were classified as representing expert strategies, indicating that expert performance was a somewhat rare and inconsistent occurrence among the students.

The ability to identify a small number of expert-like strategies within a large body of student performances may provide an opportunity to study the dynamics of complex learning at both individual and population levels as well as the emergence of medical diagnostic expertise.