Related Experiment Videos
The next Turing tests: Reimagining conceptions and measures of AI
Jose Hernandez-Orallo1,2, Katherine M Collins1,3,4, Lucy Cheke1,5
1Institute for Technology and Humanity, University of Cambridge, Cambridge, UK.
Science (New York, N.Y.)
|August 6, 2026
Summary
New methods for assessing artificial intelligence (AI) progress must prioritize diversity, collaboration, and a systemic perspective for comprehensive evaluation.
Area of Science:
- Artificial Intelligence
- Computer Science
- Evaluation Methodologies
Background:
- Current artificial intelligence (AI) progress evaluation methods may lack comprehensiveness.
- There is a need for more robust frameworks to assess AI development.
Purpose of the Study:
- To propose a new framework for evaluating artificial intelligence (AI) progress.
- To emphasize the importance of diversity, partnership, and systemicity in AI evaluation.
Main Methods:
- Conceptual framework development.
- Literature review on AI evaluation metrics.
- Analysis of systemic approaches in complex system assessment.
Main Results:
- A novel evaluation approach integrating diversity, partnership, and systemicity is proposed.
- These elements offer a more holistic view of AI progress beyond mere performance metrics.
Conclusions:
- Future AI progress evaluation should adopt a multidimensional approach.
- Incorporating diversity, partnership, and systemicity will lead to more responsible and effective AI development.
Related Concept Videos
01:16Non-equilibrium in the Cell
An important concept in studying metabolism and energy is that of chemical equilibrium. Most chemical reactions are reversible. They can proceed in both directions, releasing energy into their environment in one direction, and absorbing it from the environment in the other direction. The same is true for the chemical reactions involved in cell metabolism, such as the breaking down and building up of proteins into and from individual amino acids, respectively. Reactants within a closed system...
01:20Introduction to Cognitive Psychology
Cognitive psychology is the field of psychology dedicated to examining how people think. It attempts to explain how and why we think the way we do by studying the interactions among human thinking, emotion, creativity, language, and problem-solving, as well as other cognitive processes. Cognitive psychology studies how information is processed and manipulated in remembering, thinking, and knowing.
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
01:24Natural and Artificial Concepts
In psychology, concepts can be divided into two categories: natural and artificial. Natural concepts are formed through direct or indirect experiences. For example, consider the concept of snow. If you live in a place with regular snowfall, such as Essex Junction, Vermont, you know snow through direct experiences. You’ve seen it fall, touched it, shoveled it, and played in it. You recognize its texture, appearance, and even its smell. In contrast, if you live on an island like Saint Vincent in...
01:23Binet's Contribution to Measures of Intelligence
Alfred Binet, along with his student Théophile Simon, was tasked by the French Ministry of Education in 1904 to create a method for identifying students who struggled to learn through conventional classroom instruction. This initiative aimed to address overcrowding by placing such students in specialized schools. Binet and Simon developed an intelligence test comprising 30 tasks, ranging from simple commands, like touching one's nose or ear, to more complex tasks, such as drawing designs from...
01:29Measures of Intelligence
Psychologists measure intelligence by using standardized tests that produce a score known as the intelligence quotient or IQ. To understand IQ tests, it's important to recognize the key principles behind their construction: validity, reliability, and standardization.
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this; it...
Validity refers to how well a test measures what it claims to measure. An intelligence test should accurately assess intelligence rather than another characteristic, like anxiety. Criterion validity is one way to evaluate this; it...