Perceived humanness and empathy are dissociated in relationship advice generated by a large language model

Differentiating generated and human-written content is increasingly difficult. We examine how incentives to convey humanness and task characteristics shape perceptions of textual content across five studies. In Studies 1-2, humans and GPT-4 write relationship advice or relationship descriptions with or without instructions to appear human. New participants judge each text's source. Instructions to appear human increase perceived humanness only for GPT-4, reducing the human advantage. Study 3 shows that these effects persist under instructions to avoid sounding like an AI model and replicate across GPT-4 and GPT-4o. Study 4 demonstrates that GPT models can produce empathy without humanness and humanness without empathy. Study 5 shows that GPT-4 increases perceived humanness by adopting informal, conversational language and other surface-level stylistic cues, whereas human writers show little adjustment. Together, the findings indicate that perceived humanness depends on incentives to appear human and the context in which writing is produced.