Effects of Contributor Experience on the Quality of Health-Related Wikipedia Articles

Consulting the Internet for health-related information is a common and widespread phenomenon, and Wikipedia is arguably one of the most important resources for health-related information. Therefore, it is relevant to identify factors that have an impact on the quality of health-related Wikipedia articles.

In our study we have hypothesized a positive effect of contributor experience on the quality of health-related Wikipedia articles.

We mined the edit history of all (as of February 2017) 18,805 articles that were listed in the categories on the portal health & fitness in the English language version of Wikipedia. We identified tags within the articles' edit histories, which indicated potential issues with regard to the respective article's quality or neutrality. Of all of the sampled articles, 99 (99/18,805, 0.53%) articles had at some point received at least one such tag. In our analysis we only considered those articles with a minimum of 10 edits (10,265 articles in total; 96 tagged articles, 0.94%). Additionally, to test our hypothesis, we constructed contributor profiles, where a profile consisted of all the articles edited by a contributor and the corresponding number of edits contributed. We did not differentiate between rollbacks and edits with novel content.

Nonparametric Mann-Whitney U-tests indicated a higher number of previously edited articles for editors of the nontagged articles (mean rank tagged 2348.23, mean rank nontagged 5159.29; U=9.25, P<.001). However, we did not find a significant difference for the contributors' total number of edits (mean rank tagged 4872.85, mean rank nontagged 5135.48; U=0.87, P=.39). Using logistic regression analysis with the respective article's number of edits and number of editors as covariates, only the number of edited articles yielded a significant effect on the article's status as tagged versus nontagged (dummy-coded; Nagelkerke R² for the full model=.17; B [SE B]=-0.001 [0.00]; Wald c² [1]=19.70; P<.001), whereas we again found no significant effect for the mere number of edits (Nagelkerke R² for the full model=.15; B [SE B]=0.000 [0.01]; Wald c² [1]=0.01; P=.94).

Our findings indicate an effect of contributor experience on the quality of health-related Wikipedia articles. However, only the number of previously edited articles was a predictor of the articles' quality but not the mere volume of edits. More research is needed to disentangle the different aspects of contributor experience. We have discussed the implications of our findings with respect to ensuring the quality of health-related information in collaborative knowledge-building platforms.