What influences students' abilities to critically evaluate scientific investigations?

Area of Science:

• Science Education
• Higher Education Pedagogy
• Undergraduate Science Instruction

Background:

• Critical thinking is a key learning objective in undergraduate biology and physics.
• Assessing critical thinking in science courses presents significant challenges for educators.
• Existing methods for evaluating students' critical thinking skills are varied and debated.

Purpose of the Study:

• To evaluate the effectiveness of different assessment question formats in probing undergraduate students' critical thinking skills.
• To investigate how students critically evaluate research studies when comparing two studies versus evaluating one study individually.
• To determine if individual evaluation questions are necessary to elicit critical thinking during comparative analysis.

Main Methods:

• Utilized two standardized critical thinking instruments: the Biology Lab Inventory of Critical Thinking in Ecology (Eco-BLIC) and the Physics Lab Inventory of Critical Thinking (PLIC).
• Analyzed over 3000 student responses from more than 20 institutions.
• Compared student critical thinking performance on individual evaluation tasks versus compare-and-contrast tasks related to experimental design and data quality.

Main Results:

• Students demonstrated higher levels of critical evaluation when comparing two research studies than when assessing a single study.
• Compare-and-contrast questions alone are sufficient to elicit critical thinking.
• Including individual evaluation questions alongside compare-and-contrast questions did not significantly alter student responses or critical thinking engagement.

Conclusions:

• Compare-and-contrast questions are a more effective method for assessing critical thinking in undergraduate biology and physics.
• Introductory science instructors should integrate more compare-and-contrast questions focused on experimental design into their assessments.
• Assessment design significantly influences the elicitation of critical thinking skills in undergraduate science students.