Do first impressions matter? Comparing overall impression vs. individual item scores to make informed decisions on...

The feature matching assessment process is the gold standard in identifying effective communication systems for individuals with complex communication needs. When considering an app for a user, what is unclear is if a rater should evaluate the app by assigning an overall impression rating or individual items scores. Thus, the aim of this study was to compare the overall impression scores with individual item scores of augmentative and alternative communication (AAC) apps on a team created AAC app rating scale.

Usability testing was used to compare raters' average scores across types of communication app categories, and average item and overall impression scores. The AAC app rating scale was evaluated in two phases, the pilot (pilot raters) and testing phase (novice raters). The goal was to determine if the AAC app rating scale could be used across raters and identify if differences existed among raters.

Results indicated that overall impression scores were more reliable among pilot raters than they were among novice raters. Overall impressions scores were also 'higher' than the average item score across the types of communication app categories.

Findings suggest that lower scores on the overall impression and item scores of the apps may not define the capabilities of an app; but rather, differences may be due to the characteristics of each app and how the app aligns to the user. Practical implications for future practices are outlined.