Text understanding in GPT-4 versus humans

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Natural Language Processing

Background:

• Assessing artificial intelligence (AI) capabilities in understanding human language is crucial.
• GPT-4 is a leading AI model with advanced natural language processing abilities.

Purpose of the Study:

• To evaluate GPT-4's text comprehension abilities compared to human performance.
• To investigate GPT-4's capacity for inference and generalization in textual understanding.

Main Methods:

• Administered a standardized discourse comprehension test to GPT-4 and human participants.
• Utilized more challenging text passages, including those from university admission tests.
• Analyzed GPT-4's performance for indicators of genuine understanding, such as inference and generalization.

Main Results:

• GPT-4 performed comparably to humans on a standard comprehension test.
• GPT-4 significantly outperformed high school and university students on more difficult texts.
• GPT-4 exhibited key signatures of understanding, including generalization and inference.

Conclusions:

• GPT-4 demonstrates a high level of text comprehension, comparable to humans.
• GPT-4 shows advanced capabilities in inferring and generalizing information from text.
• The findings suggest AI systems are approaching human-level understanding in complex textual tasks.