Inference for kappas for longitudinal study data: applications to sexual health research

Analysis of instrument reliability and rater agreement is used in a wide range of behavioral, medical, psychosocial, and health-care-related research to assess psychometric properties of instruments, consensus in disease diagnoses, fidelity of psychosocial intervention, and accuracy of proxy outcomes. For categorical outcomes, Cohen's kappa is the most widely used index of agreement and reliability. In many modern-day applications, data are often clustered, making inference difficult to perform using existing methods. In addition, as longitudinal study designs become increasingly popular, missing data have become a serious issue, and the lack of methods to systematically address this problem has hampered the progress of research in the aforementioned fields. In this article, we develop a novel approach based on a new class of kappa estimates to tackle the complexities involved in addressing missing data and other related issues arising from a general multirater and longitudinal data setting. The approach is illustrated with real data in sexual health research.