CIDER: Context-sensitive polarity measurement for short-form text.

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Sentiment Analysis

Background:

• General sentiment analysis methods often fail to capture contextual nuances in short texts.
• The valence of sentiment-laden words can vary significantly based on their context (e.g., 'active lifestyle' vs. 'active volcano').

Purpose of the Study:

• To present CIDER (Context Informed Dictionary and sEmantic Reasoner), a new algorithm for context-sensitive linguistic analysis.
• To demonstrate CIDER's superior performance compared to existing generalist unsupervised sentiment analysis techniques.

Main Methods:

• CIDER infers the valence of sentiment-laden terms from the entire corpus.
• It then uses these context-specific valences to score individual texts.
• The algorithm's applicability extends beyond sentiment analysis to other linguistic scales.

Main Results:

• CIDER significantly outperforms state-of-the-art generalist unsupervised sentiment analysis methods on a large dataset of weather-related tweets.
• A case study on UK gender data identified highly gendered and sentiment-laden days, showcasing CIDER's versatility.

Conclusions:

• CIDER offers a robust solution for context-sensitive sentiment analysis, addressing limitations of general-purpose methods.
• The algorithm is effective for analyzing various linguistic scales and has been released as an open-source Python package.