Related Experiment Video
Updated: Aug 18, 2026
Assessment and Evaluation of the High Risk Neonate: The NICU Network Neurobehavioral Scale
Published on: August 25, 2014
Anticonvulsant teratogenesis 4: inter-rater agreement in assessing minor physical features related to anticonvulsant
J B Carlin1, L M Ryan, E A Harvey
1Clinical Epidemiology and Biostatistics Unit, Royal Children's Hospital Research Institute, Parkville, Victoria 3052, Australia.
Insights
Inter-rater agreement for minor physical features in infants exposed to maternal anticonvulsant use was modest. More experienced research assistants showed better agreement with physicians, highlighting the need for thorough training and multiple assessments.
Area of Science:
- Perinatal epidemiology
- Developmental toxicology
- Clinical genetics
Background:
- Assessing minor physical anomalies in newborns is crucial for understanding fetal effects of maternal exposures.
- Maternal anticonvulsant use during pregnancy is associated with a range of potential fetal effects.
- Inter-rater reliability in assessing these subtle features requires careful evaluation.
Purpose of the Study:
- To evaluate inter-rater agreement in assessing minor physical features in infants.
- To investigate agreement in a cohort study of fetal effects from maternal anticonvulsant use.
- To explore factors influencing agreement using statistical modeling.
Main Methods:
- Infants were assessed by a blinded pediatrician/teratologist and an unblinded research assistant.
- Agreement was measured using kappa statistics and log-linear modeling.
- Log-linear models were used to examine covariate effects on agreement.
Main Results:
- While overall agreement was high in proportion (80-90%), kappa values were modest (0.2-0.5) due to low anomaly prevalence.
- Log-linear modeling revealed significant variation in agreement patterns among research assistants.
- Increased research assistant experience correlated with higher agreement with physicians.
Conclusions:
- Findings have implications for clinical practice and epidemiologic research on fetal anticonvulsant effects.
- Emphasizes the critical need for comprehensive staff training on anomaly definitions.
- Underscores the importance of employing multiple, independent assessments for subtle anomalies.
Background:
We report on inter-rater agreement in the assessment of newborn infants with respect to a range of minor physical features in a cohort study of the fetal effects of maternal anticonvulsant use during pregnancy.
Methods:
Infants from three groups (exposed to anticonvulsants, seizure history but no medication exposure, and unexposed controls) were examined by both a pediatrician/teratologist, who was blinded with respect to the mother's exposure status, and an unblinded research assistant. Agreement on assessments for selected anomalies associated with anticonvulsant therapy was measured by kappa-statistics, as well as by more sensitive log-linear modeling techniques, which allow examination of possible covariate effects on the strength of agreement. Although the physician and research assistant agreed on a high proportion of cases (80-90%), kappa values were modest (0.2-0. 5), partly because of the low prevalence of the anomalies considered. To explore how agreement varies within subgroups, we used recently developed methods for studying agreement based on log-linear models.
Results:
Log-linear modeling indicated that there was substantial variation in pattern of agreement between different individual research assistants but that other factors (e.g., exposure category, sex, and birthweight) did not appear to be related to agreement. Our results suggest that research assistants with more experience showed the highest degree of agreement with the physicians.
Conclusions:
Our results have implications for both clinical practice and epidemiologic research and underline the importance of thorough training of staff in the definitions to be used and also the need for multiple independent assessments of these subtle anomalies.
