Related Experiment Video
Updated: May 10, 2025
03:14
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
462
Can Generative AI and ChatGPT Break Human Supremacy in Mathematics and Reshape Competence in Cognitive-Demanding
1Department of Mathematics Education, Faculty of Education, Nevsehir Hacı Bektas Veli University, 50300 Nevsehir, Türkiye.
Journal of Intelligence
|April 25, 2025
Summary
Generative AI tools like ChatGPT-4o and GPT-4 show strong performance on math assessments, often surpassing student scores. While they generally improve with cognitive demand, this effect was less pronounced at the 12th-grade level.
Area of Science:
- Artificial Intelligence
- Cognitive Science
- Educational Assessment
Background:
- Generative AI tools are increasingly explored for cognitive support.
- Understanding AI's impact on problem-solving and learning is crucial.
- The National Assessment of Educational Progress (NAEP) mathematics assessments provide a benchmark for student performance.
Purpose of the Study:
- To evaluate the performance of ChatGPT-4o and GPT-4 on NAEP mathematics assessments.
- To analyze AI performance in relation to cognitive complexity and demands.
- To compare AI capabilities with student performance across different grade levels.
Main Methods:
- Sixty NAEP mathematics assessment tasks were analyzed based on cognitive complexity.
- ChatGPT-4o and GPT-4 generated responses to these tasks.
- AI responses were scored using NAEP's established criteria.
- Performance was analyzed using average student scores and item-wise response percentages.
Main Results:
- ChatGPT-4o and GPT-4 outperformed most students on individual NAEP mathematics items.
- Higher cognitive demand correlated with a need for higher performance scores to answer correctly.
- This trend was consistent across 4th, 8th, and 12th grades.
Conclusions:
- Generative AI models demonstrate significant potential in mathematics problem-solving.
- AI performance shows sensitivity to cognitive demands, though with grade-level variations.
- Further research is needed to understand AI's nuanced interaction with complex cognitive tasks at higher education levels.
Keywords:
AI-driven educationChatGPTNAEPcognitive loadgenerative artificial intelligencemathematics assessmentproblem-solvingMore Related Videos
10:26Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
Published on: September 11, 2021
3.8K
09:43Reversible Cooling-induced Deactivations to Study Cortical Contributions to Obstacle Memory in the Walking Cat
Published on: December 11, 2017
6.9K
Related Concept Videos
01:10Gauss's Law: Problem-Solving
1.6K
Gauss's law helps determine electric fields even though the law is not directly about electric fields but electric flux. In situations with certain symmetries (spherical, cylindrical, or planar) in the charge distribution, the electric field can be deduced based on the knowledge of the electric flux. In these systems, we can find a Gaussian surface S over which the electric field has a constant magnitude. Furthermore, suppose the electric field is parallel (or antiparallel) to the area...
1.6K
01:12Theorems of Pappus and Guldinus: Problem Solving
664
Pappus and Guldinus's theorems are powerful mathematical principles that are used for finding the surface area and volume of composite shapes. For example, consider a cylindrical storage tank with a conical top. Finding the surface area or volume can be challenging for such complex shapes. These theorems are particularly useful in calculating the volume and surface area of such systems. Here, the cylindrical storage tank with a conical top can be broken down into two simple shapes: a...
664
01:20Introduction to Cognitive Psychology
230
Cognitive psychology is the field of psychology dedicated to examining how people think. It attempts to explain how and why we think the way we do by studying the interactions among human thinking, emotion, creativity, language, and problem-solving, as well as other cognitive processes. Cognitive psychology studies how information is processed and manipulated in remembering, thinking, and knowing.
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
230
01:17Cognitivism
1.3K
Cognitive psychology emerged as a significant field in the mid-20th century. It focused on understanding humans' internal mental processes. This approach emphasizes how people perceive, remember, think, and solve problems—elements critical to human cognition.
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
Previously dominated by behaviorism, which prioritized observable behaviors and largely ignored mental processes, psychology transformed in the 1950s. Cognitive psychologists argue that understanding how we think and process...
1.3K
01:16Non-equilibrium in the Cell
4.1K
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
4.1K
01:21Cognitive Learning
94
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
94