Structured human-LLM interaction design reveals exploration and exploitation dynamics in higher education content...

Area of Science:

• Artificial Intelligence in Education
• Human-Computer Interaction
• Information Science

Background:

• Large Language Models (LLMs) offer novel approaches to studying information foraging.
• Pedagogical content creation is a key application area for LLMs in education.
• Understanding how users interact with LLMs for information seeking is crucial.

Purpose of the Study:

• To investigate how LLM technology influences information foraging behavior in pedagogical content creation.
• To examine the role of computational-thinking skills in shaping LLM-driven foraging.
• To explore the impact of structured prompt-crafting on user behavior.

Main Methods:

• A study involving 25 doctoral students in an Artificial Intelligence in Education course.
• Utilized editable prompt templates and socially-sourced keywords to guide prompt crafting.
• Administered a Computational Thinking survey to assess participants' skills and traits.

Main Results:

• LLM use influenced participants towards exploration (generating novel information) and exploitation (focusing on specific content).
• Social cues enhanced exploration of diverse information, while exploitation narrowed focus to AI content.
• Computational thinking traits, such as cooperativity and critical thinking, influenced content exploitation and reliance on personal interests.

Conclusions:

• LLM-driven educational tools can be designed to foster both broad information exploration and focused content exploitation.
• Computational thinking skills play a significant role in mediating user interaction with LLMs for educational purposes.
• Findings have implications for the design and implementation of future AI-powered educational technologies.