Lifestyle and socio-economic status--verification of data reliability from questionnaire surveys

The study of reliability was part of a project funded by the Grant Agency of the Czech Ministry of Health. The repeatability of answers, 180 questionnaires being sent again after a six week interval, was assessed by two methods: by the percentages of agreement and using the Kappa index. We evaluated 61 questions, which were divided into five sections (general questions, questions on employment, lifestyle, health, and personality). The agreement rate varied from 46% to 100% and the value of the Kappa index from -0.01 to 1. The agreement performed by the Kappa index was divided into 4 groups (< or = 0.4--poor; 0.41-0.6--average; 0.61-0.80--good; 0.81-1 almost perfect). The agreement across our questionnaire was poor in 6.6% of the questions, average--in 31.1%, good--in 45.9% and almost perfect--in 16.4% of the questions. Next we divided the questions by their content into two groups--the factual questions (group 1) and questions where answers contain an evaluative or motivational element (group 2). Significant differences (p < 0.001) were found between groups 1 and 2 in the agreement (86.8%; 72.1%) and the Kappa index (0.73; 0.48). In questions where agreement of answers is evaluated as average or even weak we used three types of solution: a) substitution by the analogous questions with a higher agreement, b) aggregation of the answers if possible, or c) the questions were not used for the analyses. These important results will be considered in further analyses of the data and in the interpretation of the study results. The detailed information about the whole project and the results has already been published or presented are available on the following web site: www.zuova.cz/projekty/ses.php.