Related Experiment Video
Updated: Jun 2, 2026
11:21
Methodology for Establishing a Community-Wide Life Laboratory for Capturing Unobtrusive and Continuous Remote Activity and Health Data
Published on: July 27, 2018
Recent reliability reporting practices in Psychological Assessment: recognizing the people behind the data.
Carlton E Green1, Cynthia E Chen, Janet E Helms
1Institute for the Study and Promotion of Race and Culture, Boston College, 318 Campion Hall, Chestnut Hill, MA 02467, USA. isprc@bc.edu
Psychological Assessment
|April 27, 2011
Summary
Researchers often misinterpret reliability coefficients, which are sample-specific statistics, not scale properties. Improved reporting and analysis of internal consistency are crucial for accurate psychological measurement and interpretation.
Area of Science:
- Psychological Measurement
- Psychometrics
- Research Methodology
Background:
- Reliability coefficients are sample-specific statistics, not inherent properties of measures.
- The alpha-as-data perspective emphasizes analyzing reliability data for specific samples and subgroups.
- Previous research highlights issues in reporting and interpreting reliability.
Purpose of the Study:
- To evaluate contemporary and historical reliability reporting practices in psychological research.
- To demonstrate the consequences of misinterpreting and misusing reliability data.
- To advocate for improved practices in reporting, analysis, and interpretation of internal consistency.
Main Methods:
- Systematic review of reliability reporting in Psychological Assessment articles across three decades (1989, 1996, 2006).
- Analysis of archival data to illustrate the impact of reliability data misconstrual.
- Examination of reliability coefficient variations across demographic subgroups (e.g., race, gender).
Main Results:
- Reliability reporting practices have not improved over time and often deviate from best practices.
- Misinterpretation of reliability data can lead to significant repercussions in research.
- Reliability coefficients differ across racial/ethnic and gender subgroups, impacting measure suitability.
- Preliminary data analyses are essential to ensure assumptions for reliability analyses are met.
Conclusions:
- The alpha-as-data perspective necessitates careful consideration of measure choice and result interpretation.
- Adopting better practices in internal consistency analysis and reporting is vital for valid psychological research.
- Recognizing the variability of reliability data across subgroups is crucial for ethical and accurate research.
Related Concept Videos
01:29Reliability and Validity
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
01:52Regression Toward the Mean
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when researchers try to extrapolate results...
01:22Self-Report Tests of Personality
Self-report inventories are objective personality assessments that use multiple-choice items or numbered scales, typically ranging from 1 (strongly disagree) to 5 (strongly agree). They are often called Likert scales after Rensis Likert. These inventories are widely used due to their ease of administration and cost-effectiveness. One of the most prominent examples is the Minnesota Multiphasic Personality Inventory (MMPI), initially developed in the 1940s to assess abnormal personality traits.
02:30Naturalistic Observations
If you want to understand how behavior occurs, one of the best ways to gain information is to simply observe the behavior in its natural context. However, people might change their behavior in unexpected ways if they know they are being observed. How do researchers obtain accurate information when people tend to hide their natural behavior? As an example, imagine that your professor asks everyone in your class to raise their hand if they always wash their hands after using the restroom. Chances...
01:24Data Reporting and Recording
Reporting and recording are crucial in data documentation. The timely, thorough, and accurate documentation of facts is essential when recording patient data. Failure to record findings during an assessment or interpretation of a problem will result in loss of information and make the patient document unreliable. The reader is left with general impressions if the information is not specific. A recording is documenting data of the individual's health information in a traceable, secure, and...
01:56Ethics in Research
Today, scientists agree that good research is ethical in nature and is guided by a basic respect for human dignity and safety. However, this has not always been the case. Modern researchers must demonstrate that the research they perform is ethically sound.