Patterns of observer error in scoring macromorphoscopic traits for population affinity.

Area of Science:

• Forensic Anthropology
• Biological Anthropology
• Bioinformatics

Background:

• Methodological revisions are vital for identifying limitations and biases in scientific methods, ensuring reliable results.
• The subjective nature of non-metric methods raises concerns about trait scoring variation and its impact on biological parameter classification.
• Understanding observer variability in macromorphoscopic trait scoring is crucial for accurate population affinity assessments.

Purpose of the Study:

• To investigate the influence of observer experience, method familiarity, and statistical approaches on the repeatability of cranial macromorphoscopic traits.
• To assess the impact of observer factors on the reliability of classifying population affinity using skeletal traits.

Main Methods:

• Seventeen macromorphoscopic traits were scored on 10 crania by five observers with diverse experience levels.
• Intra-observer and inter-observer agreement were calculated using kappa statistics.
• A group discussion and subsequent rescoring were conducted to evaluate improvements in agreement.

Main Results:

• Intra-observer agreement varied from moderate to perfect; inferior nasal margin, nasal bone shape, and nasal overgrowth showed the lowest agreement.
• Inter-observer repeatability ranged from poor to substantial agreement, with slight improvements observed after procedural clarification.
• Familiarity with specific traits and scoring procedures positively impacted consistency, more so than general observer experience.

Conclusions:

• Method-specific training is essential for consistent application of macromorphoscopic traits in population studies.
• Statistical approach selection, including weighting methods like Cohen's kappa, significantly affects perceived trait reliability.
• Practitioners must choose statistical tests appropriate for the data type of each trait to ensure accurate reliability assessments.