A novel vision transformer model produces clock drawing test scores as accurate as expert human coders

Alzheimer's disease and related dementias is a growing public health concern. The clock-drawing test, where subjects draw a clock, typically with hands showing 11:10, has been widely used for dementia screening. A limitation of including the clock-drawing test in large-scale studies is that it requires manual coding, which could result in biases if coders interpret and implement coding rules differently. This study created and evaluated an intelligent Clock Scoring system built with deep learning neural networks to automatically code clock-drawing images. We used a large, publicly available repository of clock-drawing images from the 2011-2019 National Health and Aging Trends Study (NHATS) and compared three advanced DLNN methods - ResNet101, EfficientNet and Vision Transformers - in coding clock-drawing images into binary and ordinal (0 to 5) scores. Unlike the traditional nominal classification approach, which treats all categories as unordered (e.g., cats and dogs), we introduced structured ordering into the coding system, which recognizes that higher scores reflect better representation of the clock than lower scores. We also compared deep learning-coded clock images with manual coding, using expert coding as the benchmark to evaluate accuracy. Results suggest that Vision Transformers achieves clock scoring accuracy comparable to expert human coders (weighted kappa = 0.81), outperforming both ResNet101 and EfficientNet (weighted kappa = 0.56-0.73). The ordinal coding system has the ability to allow researchers to minimize either under- or over-estimation errors. Our Vision Transformer-based coding system has been used in NHATS' annual clock-coding since 2022.