Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Language and Cognition01:27

Language and Cognition

343
Language serves as a bridge between ideas and communication, influencing how individuals perceive and interact with the world. Psychologists have long debated whether language shapes thought or vice versa. This discussion gained grip with Edward Sapir and Benjamin Lee Whorf in the 1940s, who proposed that language determines thought, a concept known as linguistic determinism. They suggested that the vocabulary and structure of a language influence how its speakers think and perceive reality.
343
  1. Home
  3. Evaluating The Strengths And Weaknesses Of Large Language Models In Answering Neurophysiology Questions.
  1. Home
  3. Evaluating The Strengths And Weaknesses Of Large Language Models In Answering Neurophysiology Questions.

Related Experiment Video

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

547

Evaluating the strengths and weaknesses of large language models in answering neurophysiology questions.

Hassan Shojaee-Mend1, Reza Mohebbati2, Mostafa Amiri1,3

  • 1Department of General Courses, Faculty of Medicine, Gonabad University of Medical Sciences, Gonabad, Iran.

Scientific Reports
|May 11, 2024

View abstract on PubMed

Summary
This summary is machine-generated.

Large language models (LLMs) show strong neurophysiology knowledge but struggle with complex reasoning. Further training is needed to improve their advanced understanding and knowledge integration in specialized scientific fields.

Keywords:
Bloom’s taxonomyEvaluationLarge language modelsNeurophysiology

More Related Videos

Author Spotlight: Advancing Large-Scale Neural Dynamics Through HD-MEA Technology
09:44

Author Spotlight: Advancing Large-Scale Neural Dynamics Through HD-MEA Technology

Published on: March 8, 2024

4.7K
Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

7.5K

Related Experiment Videos

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

547
Author Spotlight: Advancing Large-Scale Neural Dynamics Through HD-MEA Technology
09:44

Author Spotlight: Advancing Large-Scale Neural Dynamics Through HD-MEA Technology

Published on: March 8, 2024

4.7K
Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

7.5K

Area of Science:

  • Neuroscience and Artificial Intelligence

Background:

  • Large language models (LLMs) exhibit advanced natural language processing skills.
  • Assessing LLM proficiency in specialized scientific domains like neurophysiology is vital for research and clinical applications.

Purpose of the Study:

  • To evaluate and compare the effectiveness of LLMs in answering neurophysiology questions in English and Persian.
  • To analyze LLM performance across various topics and cognitive levels.

Main Methods:

  • Twenty neurophysiology questions were designed, covering four topics and two cognitive levels.
  • Physiologists scored LLM-generated essay answers on a 0-5 scale.
  • Statistical and qualitative analyses identified performance variations and reasoning gaps.

Main Results:

  • LLMs achieved a good overall performance (mean score 3.87/5) with no significant differences between languages or cognitive levels.
  • Performance was highest for motor system topics (4.41/5) and lowest for integrative topics (3.35/5).
  • Qualitative analysis revealed limitations in reasoning, prioritization, and knowledge integration.

Conclusions:

  • LLMs demonstrate general proficiency in neurophysiology but face challenges with advanced reasoning and integrating complex knowledge.
  • Targeted training can enhance LLMs' capabilities in knowledge gaps and causal reasoning.
  • Ongoing domain-specific evaluations are essential to track LLM advancements in science.