Machine Learning for Identifying Emotional Expression in Text: Improving the Accuracy of Established Methods

Expression of emotion has been linked to numerous critical and beneficial aspects of human functioning. Accurately capturing emotional expression in text grows in relevance as people continue to spend more time in an online environment. The Linguistic Inquiry and Word Count (LIWC) is a commonly used program for the identification of many constructs, including emotional expression. In an earlier study (Bantum & Owen, 2009) LIWC was demonstrated to have good sensitivity yet poor positive predictive value. The goal of the current study was to create an automated machine learning technique to mimic manual coding. The sample included online support groups, cancer discussion boards, and transcripts from an expressive writing study, which resulted in 39,367 sentence-level coding decisions. In examining the entire sample the machine learning approach outperformed LIWC, in all categories outside of Sensitivity for negative emotion (LIWC Sensitivity = .85; Machine Learning Sensitivity = .41), although LIWC does not take into consideration prosocial emotion, such as affection, interest, and validation. LIWC performed significantly better than the machine learning approach when removing the prosocial emotions (p = <.0001). The sample over-represented examples of emotion that fit into the overarching category of positive emotion. Remaining work is needed to create more effective machine learning features for codes that are thought to be important emotionally but were not well represented in the sample (e.g., frustration, contempt, and belligerence), and Machine Learning could be a fruitful method for continued exploration.