The IPCS Collaborative Study on Neurobehavioral Screening Methods: VI. Agreement and reliability of the data

The IPCS Collaborative Study on Neurobehavioral Screening Methods was undertaken to determine the intra- and inter-laboratory reliability of a functional observational battery (FOB) and an automated assessment of motor activity in eight laboratories world-wide. The effects of seven chemicals (acrylamide, bis-acrylamide, p,p'-DDT, lead acetate, parathion, toluene, and triethyl tin) were studied during two dosing regimens: single-dose and four-week repeated dosing. All participating laboratories generally could detect and characterize the effects of known neurotoxicants, even though there were some differences in outcome on specific endpoints. The results were further evaluated to assess the agreement across laboratories in the dose-response data at the expected times of maximal effect (time of peak effect for the single-dose studies, and during or at the end of dosing for repeated-exposure studies). Percent agreement was calculated as the percentage of laboratories agreeing on an outcome (whether it be a significant dose effect or not). As an alternative approach, slopes of the dose-response functions were calculated, and reliability of those slope estimates across laboratories and chemicals was determined. Reliability was defined as the degree of agreement across laboratories (intraclass correlation coefficient) of the dose-response slopes within and between chemicals. These reliability estimates were calculated for each domain and for each endpoint. Relative reliability of the endpoints was evaluated, and hypotheses concerning the influence of outlying data were tested. The data clearly showed that reliability was not influenced by the objectivity or subjectivity of the test measure. Thus these data provide additional information regarding the reliability and robustness of the tests across the participating laboratories.