One Size Doesn't Fit All: Using Factor Analysis to Gather Validity Evidence When Using Surveys in Your Research

Area of Science:

• Educational Research
• Psychometrics
• STEM Education

Background:

• Measurement quality is paramount across scientific disciplines.
• Survey validity evidence is often overlooked in educational research.
• Rigorous research necessitates context-specific validation of survey instruments.

Purpose of the Study:

• To review essential validity aspects for survey use in research.
• To detail factor analysis as a method for collecting validity evidence.
• To illustrate factor analysis using a specific instrument validation in STEM education.

Main Methods:

• Review of validity concepts relevant to survey research.
• Description of factor analysis for exploring/confirming item relationships and identifying survey dimensions.
• Application of factor analysis to validate the goal endorsement instrument for undergraduate STEM students.

Main Results:

• Factor analysis provides crucial validity evidence by examining relationships between survey items.
• The study demonstrates the practical application of factor analysis in R.
• The goal endorsement instrument was validated for use with first-year undergraduate STEM students.

Conclusions:

• Understanding survey validity and employing methods like factor analysis is fundamental for rigorous educational research.
• Researchers must ensure and report validity evidence for their specific research context.
• Factor analysis is a powerful statistical tool for enhancing the quality of survey-based educational research.