Unsupervised quality estimation model for English to German translation and its application in extensive supervised...

Area of Science:

• Computational Linguistics
• Natural Language Processing
• Artificial Intelligence

Background:

• Machine translation (MT) evaluation is crucial for tracking progress.
• Conventional metrics rely on reference translations, which are expensive and can be language-biased.
• Existing methods often lack linguistic features or use too many, impacting repeatability.

Purpose of the Study:

• To propose a novel unsupervised MT evaluation metric.
• To overcome limitations of existing MT evaluation methods, including language bias and reliance on references.
• To develop a repeatable and cost-effective MT evaluation approach.

Main Methods:

• Developed an unsupervised MT evaluation metric utilizing a universal part-of-speech tagset.
• Did not require reference translations for evaluation.
• Explored the metric's performance on traditional supervised evaluation tasks.

Main Results:

• The proposed unsupervised metric demonstrated strong performance.
• Experiments showed higher correlation scores with human judgments compared to existing methods.
• The metric proved effective in both unsupervised and supervised evaluation settings.

Conclusions:

• The novel unsupervised MT evaluation metric offers a viable alternative to traditional methods.
• The use of universal part-of-speech tags addresses language bias and reduces costs.
• This approach enhances the reliability and practicality of MT evaluation.