The Strategic Prompting Scale (SPS) for Measuring Metacognitive Regulation in Human-AI Interaction

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Educational Psychology

Background:

• Generative language models (like NLMs) are increasingly used.
• Effective interaction requires strategic and metacognitive skills.
• No standardized scale existed to measure these user strategies.

Purpose of the Study:

• To develop and validate the Strategic Prompting Scale (SPS).
• To measure user application of planning, adaptation, and evaluation strategies with generative AI.
• To assess the psychometric properties of the SPS.

Main Methods:

• Exploratory factor analysis (EFA) on 187 participants.
• Confirmatory factor analysis (CFA) on 406 participants.
• Assessed internal consistency and construct validity.

Main Results:

• EFA revealed a three-factor structure (Planning, Adaptation, Evaluation).
• The 15-item SPS demonstrated good internal consistency and supported factor structure via CFA.
• SPS correlated positively with metacognition and critical thinking, and negatively with AI disengagement.

Conclusions:

• The SPS provides preliminary psychometric evidence for assessing metacognitive regulation in generative AI use.
• The scale is theory-driven and suitable for educational, professional, and research applications.
• It aids in understanding and improving self-regulated behaviors during AI interactions.