Integration of cognitive tasks into artificial general intelligence test for large models.

Area of Science:

• Artificial Intelligence
• Cognitive Science
• Computational Psychology

Background:

• Current large model evaluations lack a unified framework for assessing multidimensional intelligence.
• Existing methods rely on task-specific datasets, limiting comprehensive capability assessment.
• The rapid evolution of large models necessitates advanced evaluation methodologies.

Purpose of the Study:

• To introduce a comprehensive framework for evaluating large models using cognitive science-inspired artificial general intelligence (AGI) tests.
• To address the limitations of current task-specific evaluations by proposing a multidimensional intelligence assessment.
• To guide the targeted improvement of large models and facilitate their integration into society.

Main Methods:

• Developed a framework incorporating crystallized, fluid, social, and embodied intelligence tests.
• Adapted well-designed cognitive tests from human intelligence assessments.
• Integrated tests within an immersive virtual community to simulate real-world interactions.

Main Results:

• Proposed a novel AGI testing framework inspired by cognitive science principles.
• Advocated for adaptable test complexity aligned with large model advancements.
• Emphasized the importance of accurate result interpretation to prevent evaluation errors.

Conclusions:

• Cognitive science-inspired AGI tests offer a robust method for evaluating large model intelligence.
• This framework can guide specific improvements in large model capabilities.
• The proposed approach will accelerate the safe and effective integration of large models into human society.