TSTBench: A Comprehensive Benchmark for Text Style Transfer.

Area of Science:

• Computational Linguistics
• Natural Language Processing
• Artificial Intelligence

Background:

• Growing interest in text style transfer (TST) within computational linguistics.
• Existing evaluations for TST methods are insufficient for performance measurement and claim validation.
• Rapid advancements and diverse settings of TST methods create reproducibility challenges.

Purpose of the Study:

• Introduce a comprehensive benchmark for text style transfer (TST) called TSTBench.
• Provide a standardized protocol and codebase for evaluating TST algorithms.
• Facilitate reproducible research and accurate performance assessment in TST.

Main Methods:

• Developed TSTBench, a benchmark including a codebase with 13 state-of-the-art TST algorithms.
• Established a standardized protocol for conducting text style transfer experiments.
• Performed extensive evaluations across seven datasets, totaling over 7000 individual assessments.

Main Results:

• Conducted over 7000 evaluations using the TSTBench codebase and protocol.
• Analyzed the performance of representative baseline algorithms across diverse datasets.
• Identified insights into the TST task and its evaluation methodologies.

Conclusions:

• TSTBench provides a robust framework for evaluating text style transfer methods.
• The benchmark facilitates reproducible research and aids in understanding algorithm performance.
• Offers guidance for future research directions in text style transfer and its evaluation.