Can structure speak for understanding? A dual assessment of systems thinking for sustainability in preservice STEM...

Structural indicators extracted from concept maps, such as node counts, connection density, and cycle counts, are often used as proxies for systems thinking. However, it remains unclear whether a structurally complex concept map actually reflects deeper semantic understanding of a sustainability system. This study developed the Structural-Semantic Dual Assessment (SSDA) framework as an exploratory tool to examine whether and how structural indicators correspond to semantic understanding.

Forty-seven preservice STEM teachers with a geography background completed a concept-mapping task on the evolution of the Lop Nur human-land system. The SSDA framework assessed three dimensions of systems thinking: Network Structuring, Nonlinear Mechanism Deconstruction, and Spatiotemporal Reasoning. For each dimension, graph-theoretic structural indicators were extracted from the maps and compared with expert semantic ratings. We further used correlation analysis, principal component analysis-based structural-semantic profiling, representative case analysis, and a supplementary causal-chain completeness analysis.

Most participants identified system elements and annotated spatiotemporal information with reasonable consistency, but feedback mechanisms showed a different pattern. Some participants described feedback loops coherently in writing yet drew few or no closed cycles in their maps. The correspondence between structural indicators and semantic scores varied by dimension: structural indicators showed moderate correspondence with spatiotemporal reasoning but almost no correspondence with mechanistic understanding. Cross-classifying structural and semantic scores produced four learner profiles, revealing structural-semantic mismatches in both directions. The supplementary causal-chain completeness analysis further showed that valid mechanism chains depended on both structural support and link validity, especially for development-restoration tensions, feedback relations, and cross-spatiotemporal governance.

These findings indicate that structural indicators alone provide only a partial picture of systems thinking in concept maps. Structural, semantic, and causal-chain evidence are best interpreted side by side and dimension by dimension. As an exploratory framework, SSDA may support preservice STEM teacher education by helping participants develop professional judgment in evaluating systems thinking.