Emotion in language and speech: methodological issues in naturalistic approaches

Researchers currently seek to improve validity in speech and language studies by adopting naturalistic procedures In emotion-display research, validity is threatened by standard experimental controls which diminish the naturalism of stimuli and response ranges. We report two experiments comparing the adequacy of naturalistic with standard procedures. Experiment 1 had 158 judges code 89 samples of naturally-occurring emotional speech with free-choice emotion labels, and later with labels from a standard set. When free-choice labels were similar across judges, they were consistent with standard labels, but showed a range of intensity and contextual relevance. We recommend that future studies include wider options for judges when coding emotions. Experiment 2 compared valency ratings of words when presented in, or out of, context. Standard procedures score lexical valencies using affective dictionaries, disregarding natural contexts. Experiment 2 compared 23 judges' valency ratings of words presented individually, and later in their original context. Between 30% and 44% of words were rated differently in context (depending on the statistical significance level adopted). We concluded from Experiment 2 that, where small corpora adequately model a domain, the improved accuracy of valency rating achieved by presenting words in their natural context justifies the extra procedures required.