Toward better research practice--shortcomings decreasing the significance of epidemiological studies in the toxicological field

Neurobehavioral studies do not always gain the impact they should have, neither in the scientific nor in the regulatory field of neurotoxicology. Among others, shortcomings and inconsistencies across epidemiological studies may contribute to this situation. Examples were compiled to increase awareness of obstacles for conclusions. Meta-analyses were exploited since they sometimes allow the detection of deficits that are not obvious from individual studies. Exposure assessment, performance measures, and confounding were scrutinized among 98 primary studies included in meta-analyses on mercury, solvents, manganese and pesticides. Inconsistent and hardly comparable markers of exposure were found; figures, units or sampling periods were not always provided. The contribution of test materials to differences in test outcomes across studies could sometimes not be evaluated due to the insufficient description of the employed tests. Hypotheses for the selection of performance variables often remained undisclosed. Matching procedures prevailed with respect to the confounder age; the comparability of groups with respect to intelligence and gender remained more elusive. 8% and 16% of the studies did not even mention confounding from intelligence and gender, respectively. Only one third of the studies provided adjusted means for group comparisons; the proportion was slightly larger for studies published 2000-2010. While 50% of the studies considered confounders for their dose-response assessment, only 29% reported results for the total of test variables. The outlined deficits impede, among others, the assessment of exposure-effect relationships and confounding across studies; thereby they limit the use of the studies for toxicological risk assessment and future prevention. Some shortcomings also impede a deeper insight into the mechanisms of toxicity: tests like the Digit Symbol show that something is affected, but not what is affected. Thorough description of measures employed is among the first consequences from the data. The consideration of mechanistic insights from research on animals and neurobiology may further help to increase the significance of epidemiological studies.