Concept Maps to Assess System Understanding: Are Graphical Explanations More Accurate than Verbal Ones?

Solving problems in a technical system usually requires people to understand its functioning on different levels of abstraction (i.e., goals, functions, components, characteristics) that are connected via means-ends links. We combined this abstraction hierarchy with concept mapping to assess people's understanding of system functioning. The present study examines the benefits and drawbacks of the proposed method by comparing it to a viable alternative, namely verbal explanation. Using a set of pre-defined concepts, twenty-four participants explained the functioning of two everyday systems: one graphically by constructing a concept map and one verbally. The verbal explanations were subsequently transformed into concept maps by the authors. Compared to verbal explanations, participant-constructed concept maps contained a higher proportion of functional propositions, and lower proportions of structural, temporal, general, and other propositions. Contrary to our expectations, there was no difference regarding the accuracy of functional propositions. Even though participants needed far less time to explain system functioning verbally, our results indicate that concept mapping is better suited to assess functional system understanding. We discuss how this benefit relates to the cognitive processes during concept mapping, and how the method needs to be adapted to assess functional understanding of more complex systems.