Large language model attributes and perceived moral agency of artificial moral advisors

Large Language Models (LLMs) are increasingly used as conversational artificial moral advisors capable of providing ethical guidance, moral reflection, and recommendations regarding complex moral dilemmas. Despite their ability to generate coherent and persuasive moral discourse, LLMs lack genuine moral understanding, intentionality, and reflective judgment, raising concerns that users may overattribute moral agency to these systems. Drawing on Theory of Mind, AI Moral Agency, and the Godspeed framework, this study investigates how perceived LLM attributes-namely anthropomorphism, animacy, likeability, and intelligence-are associated with users' perceptions of agency and morality in the context of artificial moral advisors.

A cross-sectional survey was conducted with 758 valid respondents who were presented with vignette-based moral dilemmas relevant to young adults and instructed to imagine seeking advice from LLM-based conversational agents.

Perceived agency was positively associated with perceived morality, suggesting that users are more likely to regard LLM-based artificial moral advisors as morally capable when they perceive them as autonomous, intentional, and self-directed entities. Perceived LLM anthropomorphism and animacy were significantly and positively associated with perceived agency, with perceived agency fully mediating their associations with the perceived morality of artificial moral advisors. Perceived LLM likeability and intelligence were positively associated with perceived morality but were not significantly associated with perceived agency, suggesting that users may infer moral capability from social appeal and cognitive competence without necessarily attributing autonomy or intentionality to these systems.

These findings offer important theoretical and practical implications for the ethical design of artificial moral advisors, particularly regarding the need to prevent inflated perceptions of AI moral competence and authority.