Neural Machine Translation With GRU-Gated Attention Model.

Area of Science:

• Natural Language Processing
• Artificial Intelligence
• Machine Learning

Background:

• Neural machine translation (NMT) uses attention mechanisms for context vectors.
• Degenerative translations arise from similar, non-discriminatory context vectors due to invariant source representations.

Purpose of the Study:

• To propose a novel Gated Recurrent Unit (GRU)-gated attention (GAtt) model for NMT.
• To address the issue of invariant source representations and improve context vector discrimination.

Main Methods:

• Developed a Gated Recurrent Unit (GRU)-gated attention (GAtt) model for NMT.
• Incorporated previous decoder states into source representations via a GRU.
• Proposed a variant GAtt model with swapped input order to the GRU.

Main Results:

• GAtt models significantly improved performance over vanilla attention-based NMT.
• Demonstrated enhanced discrimination in context vectors and source representations.
• Showcased effectiveness in mitigating overtranslation in machine translation tasks.

Conclusions:

• The proposed GAtt models effectively generate discriminative context vectors.
• GAtt enhances source representation dynamics, leading to improved NMT quality.
• This approach successfully addresses the challenge of overtranslation in NMT.