Multi-task learning for toxic comment classification and rationale extraction

Social media content moderation is the standard practice as on today to promote healthy discussion forums. Toxic span prediction is helpful for explaining the toxic comment classification labels, thus is an important step towards building automated moderation systems. The relation between toxic comment classification and toxic span prediction makes joint learning objective meaningful. We propose a multi-task learning model using ToxicXLMR for bidirectional contextual embeddings of input text for toxic comment classification, and a Bi-LSTM CRF layer for toxic span or rationale identification. To enable multi-task learning in this domain, we have curated a dataset from Jigsaw and Toxic span prediction datasets. The proposed model outperformed the single task models on the curated and toxic span prediction datasets with 4% and 2% improvement for classification and rationale identification, respectively. We investigated the domain adaptation ability of the proposed MTL model on HASOC and OLID datasets that contain the out of domain text from Twitter and found a 3% improvement in the F1 score over single task models.