The Role of Rating Valence in AI Skin Cancer App Acceptance: Eye-Tracking and Questionnaire Study

Artificial intelligence-based skin cancer screening apps (AISCSAs) offer diagnostic potential but face limited adoption. App store cues, such as ratings, may influence acceptance; yet, little is known about how users cognitively process app store information in high-stakes health contexts. To address this gap, eye-tracking was used to measure visual attention while participants evaluated a mock AISCSA app store listing.

This study aimed to test whether a single negative rating captures visual attention and whether an extended technology acceptance model (TAM) can predict behavioral intention to use (BI) AISCSAs.

Participants (N=76) evaluated a mock app store listing for an AISCSA under positive (n=42) or negative (n=34) rating conditions while their eye movements were recorded. Analyses combined fixation durations in defined areas of interest (AOIs) with self-reported measures of perceived usefulness (PU), perceived ease of use (PEOU), trust, BI, willingness to pay, and the self-rated importance of app attributes.

Normalized fixation durations (seconds per square pixel) revealed the highest attention to the description (0.166 s/px2), followed by the reviews (0.11 s/px2) and the ratings (0.04 s/px2), while the price and the data protection received the least attention. Of the 5 self-rated app attributes, only reviews correlated positively with fixation durations on the reviews-AOI (r=0.28; P=.01). Rating valence had no significant effect on gaze patterns, PU, PEOU, trust, BI, or willingness to pay (all Ps>.05). However, PEOU (P=.001), PU (P<.001), and trust (P<.001) were significantly correlated with BI.

Although the expected attentional capture effect of the negative rating was not observed, the weak or nonexistent associations between fixation durations on the AOIs and the self-rated importance of app attributes suggest that eye-tracking captures aspects of information processing that are not directly reflected in self-reported evaluations. These findings indicate that eye-tracking provides a more direct approximation of actual user behavior by revealing implicit attentional processes beyond what is captured by questionnaires. While the technology acceptance model constructs and trust predicted BI, rating valence alone did not affect acceptance or gaze behavior. In high-stakes health contexts, textual information may outweigh rating valence in driving adoption. Future research should explore conditions under which rating valence matters, including more extreme rating contrasts, variations in accompanying review texts, and the influence of individual differences such as preexisting attitudes toward artificial intelligence and levels of artificial intelligence literacy.