A Multi-Label Classification Approach for Coding Cancer Information Service Chat Transcripts.

Area of Science:

• Computational Linguistics
• Health Informatics
• Cancer Information Services

Background:

• The National Cancer Institute's (NCI) Cancer Information Service (CIS) utilizes LiveHelp, an instant messaging service, to provide cancer information to consumers.
• Information specialists (IS) manually code extensive metadata from chat sessions, a time-consuming process impacting their primary role.
• Automating this metadata coding could enhance efficiency and allow ISs to focus more on consumer support.

Purpose of the Study:

• To investigate the feasibility of automatically coding LiveHelp chat transcripts using text classification.
• To develop and evaluate machine learning models for classifying the purpose, subjects, and responses within chat sessions.
• To establish a baseline for future research in automating cancer information service transcript analysis.

Main Methods:

• Exploration of multi-label and multi-class text classification approaches.
• Training and evaluation on a dataset of approximately 673 LiveHelp chat transcripts.
• Analysis of coding accuracy using example-based and label-based F-scores.

Main Results:

• Achieved example-based F-scores of 0.67 for subjects and 0.58 for responses.
• Obtained label-based micro F-scores of 0.65 for subjects, 0.62 for responses, and 0.61 for purpose.
• Demonstrated promising initial results on a smaller corpus for automated transcript coding.

Conclusions:

• This study represents the first attempt at automatically coding LiveHelp transcripts.
• Text classification models show potential for accurately coding key metadata elements from chat sessions.
• The findings suggest a promising direction for improving the efficiency of cancer information services.