Machine Learning for Identifying Emotional Expression in Text: Improving the Accuracy of Established Methods.

Area of Science:

• Computational Linguistics
• Psychology
• Natural Language Processing

Background:

• Accurate analysis of emotional expression in text is crucial, especially in online environments.
• The Linguistic Inquiry and Word Count (LIWC) program is widely used but has limitations in positive predictive value.
• Previous research indicated LIWC's good sensitivity but poor positive predictive value for emotional constructs.

Purpose of the Study:

• To develop an automated machine learning (ML) technique to replicate manual coding of emotional expression in text.
• To compare the performance of the ML approach against the established LIWC program.
• To identify areas for improvement in ML models for emotional expression analysis.

Main Methods:

• Utilized a large dataset of 39,367 sentence-level coding decisions from online support groups, cancer discussion boards, and expressive writing studies.
• Developed and implemented an automated machine learning technique to mimic manual coding of emotional expression.
• Compared the ML approach's performance against the Linguistic Inquiry and Word Count (LIWC) program.

Main Results:

• The ML approach outperformed LIWC in most categories, except for sensitivity to negative emotion (LIWC: 0.85, ML: 0.41).
• LIWC showed significantly better performance than ML when prosocial emotions (e.g., affection, validation) were excluded (p < .0001).
• The study sample was over-represented in positive emotion examples, potentially impacting ML model generalizability.

Conclusions:

• Machine learning presents a promising avenue for analyzing emotional expression in text, potentially surpassing traditional methods like LIWC.
• Further research is needed to refine ML features for less represented emotional codes (e.g., frustration, contempt).
• The effectiveness of ML models is influenced by the representativeness of emotional categories within the training data.