Can Dental Professionals Distinguish Human-Authored From AI-Generated Scientific Abstracts? A Blinded Cross-Sectional Study

To evaluate whether dental professionals can distinguish human-authored from generative artificial intelligence (GenAI)-generated dental research abstracts under blinded conditions and to explore differences in detectability across contemporary large language models (LLMs).

In this exploratory blinded cross-sectional study, 23 dental professionals (10 general dentists, 7 specialists, 6 dental students) evaluated 60 abstracts derived from 15 published dental research articles: the 15 published abstracts and 45 GenAI-generated abstracts produced from the main text of the same articles by Claude Sonnet 4.6, GPT-5.4 and Gemini 3. Participants classified each abstract as human-authored, AI-generated or 'Unsure' and rated confidence, quality and credibility on five-point scales. Discrimination was assessed using sensitivity, specificity, balanced accuracy and signal-detection measures, with a probit mixed-effects model accounting for variability between participants, abstracts and source articles.

Of 1377 classifications, 10.0% were 'Unsure'. Balanced accuracy was 56.0% (95% CI 49.6%-62.4%) and did not differ significantly from chance (p = 0.070). In the mixed-effects model, discrimination was weak and not statistically significant (d' = 0.39, 95% CI -0.08-0.86) and participants showed no systematic response bias. Detectability did not differ between LLMs (p = 0.157) or expertise groups (p = 0.99). Confidence was not calibrated to accuracy (mean 3.58 for correct vs. 3.61 for incorrect classifications, p = 0.75) and GenAI-generated abstracts were not rated lower than human-authored abstracts in quality or credibility. Findings were consistent across sensitivity analyses.

Dental professionals showed limited ability to distinguish human-authored from fully GenAI-generated dental research abstracts, regardless of expertise or LLM and their confidence did not reflect their accuracy. These exploratory findings support explicit training in AI literacy and critical appraisal within dental education, rather than reliance on stylistic intuition.