Efficacy of ChatGPT in Cantonese Sentiment Analysis: Comparative Study

Area of Science:

• Natural Language Processing
• Computational Linguistics
• Artificial Intelligence

Background:

• Sentiment analysis presents unique challenges in Cantonese due to its linguistic distinctiveness.
• Advanced language models like ChatGPT offer potential solutions for these challenges.

Purpose of the Study:

• To evaluate the effectiveness of GPT-3.5 and GPT-4 for Cantonese sentiment analysis.
• To compare their performance against lexicon-based and machine learning methods in a real-world counseling context.

Main Methods:

• Analysis of 6169 messages from 131 Cantonese web-based counseling sessions.
• Sentiment labeling of messages using GPT-3.5 and GPT-4 with a simple prompt.
• Comparison of model performance against lexicon-based and machine learning approaches (linear regression, SVM, LSTM).

Main Results:

• GPT-4 achieved 95.3% accuracy and GPT-3.5 achieved 92.1% accuracy in sentiment classification.
• Both models significantly outperformed a lexicon-based method (37.2% accuracy) and machine learning models (66%-70.9% accuracy).

Conclusions:

• ChatGPT models demonstrate superior accuracy for Cantonese sentiment analysis.
• These findings highlight ChatGPT's utility in real-world applications like monitoring counseling services and analyzing specialized domains.