Examining Sentiment in Complex Texts. A Comparison of Different Computational Approaches

Area of Science:

• Computational linguistics
• Digital humanities
• Natural language processing

Background:

• Analyzing complex texts like literature reviews presents challenges due to nuanced and ambiguous language.
• Traditional computational sentiment analysis methods may not capture the full spectrum of sentiment in such corpora.
• A metric scale for sentiment analysis is proposed over a dichotomous scale to account for nuanced expressions.

Purpose of the Study:

• To evaluate the accuracy of different computational methods for sentiment analysis of German literature reviews.
• To compare automated and semi-automated approaches against a human-coded "gold standard".
• To provide guidance on selecting appropriate computational methods and pre-processing techniques for complex texts.

Main Methods:

• Comparison of prefabricated dictionaries, self-created dictionaries using word embeddings (pre-trained and self-trained), and a semi-automated approach.
• Sentiment prediction was benchmarked against human-coded sentiments using a metric scale.
• Analysis of word embedding methods, including coding intensity and pre-processing requirements.

Main Results:

• Prefabricated dictionaries showed low to medium correlation (r=0.32-0.39) with human-coded sentiments.
• Self-created dictionaries using word embeddings yielded lower accuracy (r=0.10-0.28).
• A semi-automated approach achieved higher correlation (r≈0.6) but required significant human coding for training.

Conclusions:

• Fully automated computational sentiment analysis is not yet reliable for complex texts like literature reviews.
• Semi-automated approaches, despite requiring human effort, demonstrate greater potential for accurate sentiment analysis in nuanced corpora.
• Researchers should carefully consider method selection and pre-processing levels when analyzing complex texts computationally.